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TOI.  It  is  suspected  that  anomaly  6080  corresponds  to  seed  WK-18 . 65 


Figure  47:  Polarizabilities  for  targets  flagged  as  high  likelihood  small  ISO  during  QC  (1-7).  Item 
(8)  shows  polarizabilities  for  an  item  not  flagged  as  high  likelihood  TOI  but  still  classified  as  a 
dig.  It  is  suspected  that  this  item  corresponds  to  seed  WK-18  (small  ISO).  The  blue  labels 


represent  best  guesses  at  corresponding  seed  IDs . 65 

Figure  48:  Polarizabilities  for  three  targets  flagged  as  high  likelihood  TOI  during  QC . 66 
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EXECUTIVE  SUMMARY 

The  Man-Portable  Vector  (MPV)  sensor  was  demonstrated  at  six  residential  properties  along 
Puako  Dr  on  Hawaii  in  December  2015  as  part  of  the  ESTCP  Live-Site  Program  for  Munitions 
Response.  This  document  reports  on  the  data  collection  and  the  analysis  that  supported  the 
digital  geophysical  mapping  and  classification  that  was  done  at  the  site. 

The  MPV  is  an  electromagnetic  induction  (EMI)  sensor  designed  for  munitions  detection  and 
classification.  The  MPV  is  a  handheld  instrument,  and  thus  represents  an  advanced  EMI  sensor 
solution  for  sites  that  are  not  ideal  for  cart  or  vehicular  based  systems.  The  MPV  supports  two 
deployment  modes:  dynamic  data  collection  along  survey  lines  to  establish  a  map  of  the  UXO 
contamination;  and  static,  cued  interrogation  of  selected  anomalies  to  acquire  high-quality  data 
for  classification. 

Prior  to  this  study,  the  technology  had  been  demonstrated  at  seven  ESTCP  demonstration 
sites:  Yuma  Proving  Ground  (2010),  Camp  Beale  (2011),  Spencer  Range  and  George  West 
(2012),  New  Boston  (2013),  Former  Waikoloa  Maneuver  Area  (WMA)  (2014),  and  Tobyhanna 
(2015).  The  technology  has  been  tested  for  detection  and  classification  with  static  and  dynamic 
data  under  multiple  environmental  conditions:  open  field,  low  and  high  density  forests  and  steep 
sided  hills. 

The  MPV  was  deployed  on  the  Puako  Drive  properties,  in  part,  due  to  its  classification 
performance  demonstrated  on  the  highly  magnetic  geologic  sites  of  the  WMA  demonstration, 
and  also  due  to  its  ability  to  survey  residential  properties  with  a  minimal  site  impact.  The  MPV 
study  focused  on  six  different  residential  properties  on  Puako  Drive,  HI.  The  Puako  site 
presented  challenges  related  to  surveying  in  a  residential  area  including:  noise  levels  that  varied 
between  properties,  amplitude  saturated  areas  related  to  infrastructure  (e.g.  buildings,  vehicles, 
reinforced  concrete  driveways,  underground  utilities)  and  overhead  tree  canopy  obscuring  GPS 
satellites. 

At  each  house,  the  MPV  was  first  used  to  acquire  a  detection  survey  and  to  create  a  digital 
geophysical  map.  Dynamic  data  were  collected  along  straight  lines  with  0.5  m  line  spacing. 
Each  house  was  treated  as  a  unique  site,  with  site  specific  noise  characterized  for  each  property, 
to  ensure  that  appropriate  target  picking  parameters  were  used.  The  dynamic  data  for  each 
property  were  immediately  analyzed,  and  data  anomalies  were  selected  for  MPV3D  cued 
measurements.  Classification  was  applied  to  the  MPV3D  cued  data  collected  at  the  six  Puako 
drive  properties  and  100%  of  detected  TOI  were  successfully  classified  with  80%  clutter 
rejection. 

All  seeds  that  were  emplaced  within  the  project  design  specifications  of  surface  to  fifteen 
centimeters  depth  were  detected  and  classified  correctly.  Four  small  ISO  seeds  were  buried 
deeper  than  fifteen  centimeters  and  were  correctly  detected  and  classified.  One  small  ISO  was 
buried  at  twenty-eight  centimeters  and  was  not  detected.  A  retrospective  analysis  of  the  dynamic 
data  surrounding  that  item  determined  this  seed  was  located  at  a  depth  below  the  detection 
capability  of  the  technology. 

At  one  of  the  properties,  the  overhead  tree  canopy  made  RTK  GPS  positioning  infeasible. 
Therefore,  anomalies  were  detected  and  flagged  in  real  time  by  using  the  “dancing  arrows” 
display  to  guide  the  MPV  over  potential  targets.  This  real-time  approach  was  successful  in 
correctly  identifying  the  8  QC  seeds  on  the  property. 
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The  demonstration  met  all  but  two  performance  objectives.  The  dynamic  production  rate  was 
less  than  the  intended  0.7  acres  per  day  as  a  consequence  of  shifting  a  significant  portion  of  the 
intended  survey  area  from  the  open  fire  break  area  to  the  more  tedious  and  difficult  residential 
properties.  A  small  number  of  dynamic  IVS  amplitudes  and  cued  polarizability  match  metrics 
were  also  outside  of  the  stated  objectives. 
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1.0  INTRODUCTION 

The  demonstration  at  the  residential  areas  on  Puako  Dr  on  Hawaii  is  one  in  the  series  of 
Environmental  Security  Technology  Certification  Program  (ESTCP)  demonstrations  of 
classification  technologies  for  Munitions  Response  (MR).  This  demonstration  was  designed  to 
investigate  the  classification  methodology  in  a  residential  environment  that  includes  utilities, 
cultural  debris  and  significant  infrastructure  in  addition  to  the  magnetic  soils  native  to  the 
Hawaiian  Islands.  The  main  objective  was  an  understanding  of  the  potential  impacts  of  surveying 
in  a  developed  area  on  geophysical  data  quality  and  the  ability  to  apply  advanced  classification 
methods.  The  project  took  place  in  December  of  2015. 

This  project  demonstrated  use  of  the  Man  Portable  Vector  (MPV)  sensor  at  six  residential 
house  lots  in  the  Puako  area.  The  MPV  is  a  new-generation  electromagnetic  induction  (EMI) 
technology  packaged  in  a  handheld  form  factor.  The  MPV  is  an  ideal  advanced  classification 
data  acquisition  solution  at  sites  with  challenging  surveying  conditions  and  can  reach  most 
human  trafficable  land  locations  at  moderate  cost  (Figure  1)  with  minimal  environmental  impact. 

The  Puako  study  followed  a  standard  two-stage  approach  used  for  classification  at  a 
munitions  response  site:  First,  a  full-coverage  survey  with  an  EMI  sensor  to  map  the  munitions 
contamination  and  locate  signal  anomalies  where  potential  TOI  might  be  located.  Second,  re¬ 
acquisition  of  selected  anomaly  locations  to  collect  EMI  data  in  cued  interrogation  mode,  where 
data  of  the  highest  quality  were  collected  for  classification. 

This  project  also  provided  the  opportunity  to  test  technology  transfer  with  the  participation  of 
the  Parsons  field  crew.  The  field  crew  was  trained  to  collect  MPV  data  in  dynamic  and  cued 
survey  modes.  In  particular,  the  field  crew  learned  how  to  operate  the  onboard  instrument 
software,  how  to  handle  the  sensor  in  detection  mode  such  that  data  gaps  are  minimized,  how  to 
recognize  instrument  malfunctions,  how  to  assess  data  quality,  and  how  to  interpret  data  displays 
and  in  field  QC  results  in  order  to  estimate  the  relative  location  of  a  buried  target. 


Figure  1:  Detection  with  the  MPV  at  one  of  the  six  residences  surveyed  in  Puako,  HI. 
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2.0  TECHNOLOGY 

The  MPY  technology  is  based  on  electromagnetic  induction  sensing  using  one  transmitter 
coil  and  multiple  vector  receivers  in  a  handheld  form  factor.  The  sensor  presented  in  this  study  is 
the  third-generation  prototype  MPV,  dubbed  MPV2,  which  was  deployed  with  the  same 
hardware  configuration  at  Spencer  Range,  Camp  George  West  (ESTCP  MR- 201 158)  and  New 
Boston  (ESCTP  MR-201228).  Cued  measurements  were  made  using  the  first-generation 
horizontal  Transmitter  coils  (MPV3D)  first  tested  at  WMA  in  2014  and  at  Tobyhanna  in  2015 
(both  under  ESCTP  MR-201228)  and  shown  in  Figure  2. 


Figure  2:  Collection  of  cued  MPV3D  data  at  one  of  the  six  residences  surveyed  in  Puako,  HI. 

2.1  TECHNOLOGY  DESCRIPTION 

The  Man  Portable  Vector  EMI  instrument  was  designed  to  (1)  be  man  portable,  such  that  it 
could  be  deployed  at  sites  where  fielding  a  cart  based  or  vehicular  based  systems  would  be 
difficult,  and  (2)  acquire  multi-static,  multi-transmitter,  and  multi-time  channel  data  suitable  for 
UXO  classification  methods.  The  MPV  consists  of  sensor  head  that  is  attached  to  fibreglass 
handle,  with  the  data  acquisition  (DAQ)  system,  batteries  and  system  electronics  located  on  the 
backpack  worn  by  the  operator  (Figure  2).  A  touch-screen  display  is  used  to  control  survey 
parameters  and  acquisition  events.  Positioning  is  derived  from  global  coordinates  obtained  with  a 
GPS  rover  and  angles  measured  with  an  Attitude  and  Heading  Reference  System  (AHRS)  sensor. 
Both  sensors  are  mounted  on  the  top  end  of  the  MPV  boom. 
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The  MPV  can  be  operated  in  either  dynamic  or  cued  surveying  modes.  In  dynamic  mode 
surveying,  the  MPV  is  used  to  produce  a  digital  geophysical  map  of  the  site  and  the  data  is  used 
to  identify  locations  of  suspected  metallic  objects.  Either  these  data  are  used  as  the  basis  for 
classification  decisions,  or  anomalous  regions  are  revisited,  and  interrogated  in  a  cued  mode. 
When  collecting  data  in  dynamic  mode,  a  50  cm  diameter  wire-loop  transmitter  inside  the  sensor 
head  generates  a  time  varying  primary  field  that  illuminates  the  subsurface.  When  collecting  in 
cued  mode,  the  MPV  is  positioned  on  top  of  a  potential  target,  and  a  pair  of  rectangular  coil 
transmitters  are  placed  on  top  of  the  sensor  head  (Figure  3).  The  three  transmitters  are  “fired” 
sequentially  to  provide  the  multiple  angles  of  illumination  of  the  target  required  for  estimating 
the  dipole  polarizabilities  used  for  classification. 


A. 


B. 


Figure  3:  MPV3D  concept  with  two  horizontal-axis  transmitters  placed  on  top  of  MPV2  head. 

A:  Side  view  of  the  3D  system  (new  transmitters  made  out  of  wood). 

B:  View  from  above  of  the  orthogonal  transmitters. 

In  either  cued  or  dynamic  survey  modes,  five  receiver  cubes  within  the  sensor  head  measures 
the  secondary  field  generated  by  eddy  currents  induced  in  metallic  targets.  Each  cube  measures 
three  orthogonal  components  of  the  secondary  field  with  three  rectangular  coils.  The  receiver 
cubes  are  arranged  in  a  “+”  pattern,  with  one  cube  at  the  center  of  the  sensor  head  and  four  cubes 
on  the  outer  edge  of  the  head.  The  multi-static  design  of  the  MPV  provides  improved  dipole 
location  capability,  relative  to  traditional  co-axial  designs. 

The  duration  of  the  transmitter  pulse  on-time,  and  the  time  window  over  which  the  receivers 
measure  the  response  is  set  within  the  data  acquisition  software  “EM3D”  developed  by  G&G 
Geosciences.  A  data  block  contains  multiple  repeats  of  the  EMI  receiver-transmit  cycle.  For 
dynamic  data  acquisition,  the  data  block  is  typically  0.1  sec,  such  that  the  sensor  can  move 
continuously  without  smearing  the  data.  There  is  a  tradeoff  between  the  duration  of  a  transmit- 
receive  cycle  and  the  amount  of  stacking  than  can  be  done  within  a  data  block.  Depending  on  the 
site  conditions  we  use  2.7,  8.3  or  25  msec  time  decay,  which  allows  respectively  9,  3  or  just  1 
full  cycle.  The  default  dynamic  acquisition  setting  is  2.7  msec,  which  allows  some  stacking  to 
reduce  noise  and  false  alarms,  while  still  retaining  some  capability  for  screening  fast  decaying 
objects.  Cued  data  acquisition  acquires  higher  quality  data,  by  using  multiple  cycles  of  target 
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excitation  and  averaging  or  stacking  the  measured  data  to  reduce  the  random  noise.  Since  the 
instrument  is  not  moving,  longer  transmit-receive  cycles  (e.g.,  8  msec  or  25  msec  time  decay) 
can  be  employed  to  capture  a  larger  portion  of  the  time  decay  rate  of  the  target  response.  The 
later  time  data  provide  additional  information  for  target  type  identification  and  is  important  for 
distinguishing  between  intact  ordnance  and  thinner  walled  shrapnel  and  cultural  debris  (Billings 
et  al.,  2007). 

The  MPY  user  interface  has  data  monitoring  capabilities.  The  recorded  data  can  be  displayed 
in  real  time,  allowing  the  operator  to  verify  data  quality.  For  example,  the  operator  can  identify 
malfunctioning  receivers  and  transmitters  by  inspecting  the  data.  A  function  test  -  i.e.  a  static 
measurement  with  a  known  item  -  can  be  immediately  inverted  and  inspected  to  ensure  that  the 
instrument  is  operating  as  expected.  The  past  and  current  sensor  location  is  displayed  on  a 
navigation  map  verify  spatial  coverage  and  global  location.  Preset  survey  points  are  also 
displayed  on  the  map  to  aid  with  surveying. 

The  MPY’s  EM3D  software  includes  an  inversion  based  in-field  method  for  determining 
accurate  positioning  on  top  of  a  target  during  cued  surveying.  Data  from  the  most  recent  cued 
sounding  are  used  to  estimate  the  location  of  a  dipole  source.  If  the  inverted  position  lies  beyond 
a  project  determined  offset  from  the  center  of  the  MPV  sensor,  the  operator  can  reposition  the 
instrument  closer  to  the  anomaly  source. 

The  sensor  requires  positioning  for  detection  and  classification.  Given  that  it  was  anticipated 
that  the  entire  site  would  have  open  sky  view,  we  used  a  Global  Positioning  System  (GPS) 
Trimble  R8  receiver  unit  and  an  Attitude  and  Heading  Reference  System  (AHRS)  XSens  MTi 
unit  that  were  attached  to  the  MPV  handle  to  provide  centimeter-level  positioning  accuracy  of 
the  MPV  sensor  head. 

2.2  TECHNOLOGY  DEVELOPMENT 

MPV  development  was  initiated  in  2005  as  part  of  the  Strategic  Environmental  Research  and 
Development  Program  (SERDP)  through  the  project  “New  Man-Portable  Vector  Time  Domain 
Electromagnetic  Induction  Sensor  and  Physically  Complete  Processing  Approaches  for  UXO 
Discrimination  Under  Realistic  Field  Conditions  (MR- 1443)”  led  by  Drs.  Kevin  O’Neill  and 
Benjamin  Barrowes.  The  first  MPV  prototype  was  built  in  2005-2006  with  David  George  of 
G&G  Sciences.  It  was  tested  in  2007  at  ERDC  in  a  laboratory  setting.  Data  analysis  showed  that 
stable  target  parameters  could  be  estimated  and  used  for  UXO  classification. 

The  SERDP  project  was  extended  in  2008  to  continue  testing  with  the  current  Black  Tusk 
Geophysics  team.  Field  trials  were  done  on  a  test  plot  to  assess  static  and  dynamic  acquisition 
mode  over  buried  targets  and  verify  that  stable  target  parameters  could  be  recovered.  The  effect 
of  magnetic  soil  on  EMI  sensors  was  investigated  with  the  MPV.  It  was  demonstrated  that  the 
particular  geometric  design  and  cube  distribution  could  be  used  to  defeat  some  of  the  adverse 
soil  effects.  The  positioning  system  was  evaluated  for  practical  field  use.  We  found  that  the 
ArcSecond  laser  ranger  was  impractical  due  to  the  requirement  to  maintain  line-of-sight  for  three 
rovers  and  tedious  calibration.  The  SERDP  project  was  further  extended  in  2009  to  test  an 
alternative  positioning  system  based  on  the  beacon  concept  and  prepare  modification  of  the 
original  MPV  prototype  for  extensive  field  deployments.  The  sensor  head  was  redesigned  with 
lighter  materials  and  a  smaller  head  diameter  to  reduce  weight  and  improve  maneuverability 


BLACK  TUSK  GEOPHYSICS 


4 


Mar.  2017 


ESTCP  MR-201228:  MPV  Demonstration  at  Puako,  HI 


Final  Report 


while  maintaining  its  expected  performance  (Lhomme,  2011b).  Fabrication  of  the  new  head 
began  under  that  SERDP  funding  extension. 

The  ESCTP  MR-201005  project  had  the  objective  to  prove  the  concept  of  classification  with 
the  MPY  at  live  sites.  The  MPV  fabrication  and  integration  of  a  new  DAQ  was  completed  before 
this  second-generation  MPV  was  demonstrated  at  Yuma  Proving  Ground  UXO  test  site  in 
October  2010.  The  technology  was  first  demonstrated  at  a  live  site  at  former  Camp  Beale  in  June 
2011  for  cued  interrogation  in  open  field  and  in  a  moderately  dense  forest.  In  ESTCP  MR- 
201158  the  MPV  was  demonstrated  at  Spencer  Range,  TN  in  June  2102  for  detection  and 
dynamic  classification  in  open  field  and  cued  interrogation  in  a  forest,  and  at  former  Camp 
George  West,  CO  in  October  2012  on  the  side  of  a  mountain  with  slopes  up  to  40%.  In  ESTCP 
MR-201228  the  technology  was  tested  in  a  dense  forest  at  the  New  Boston  Air  Force  Station  in 
August  2013,  at  WMA  in  January  of  2014  and  at  Tobyhanna  in  September  of  2015. 

2.3  ADVANTAGES  AND  LIMITATIONS  OF  THE  MPV  TECHNOLOGY 

The  MPV  is  the  only  available  handheld  sensor  that  can  acquire  multi-static,  multi- 
component  data  on  a  wide  and  programmable  time  range.  The  MPV  offers  several  key  benefits: 

-  Hand-held  form  factor:  The  MPV  can  be  deployed  at  sites  where  terrain  and  vegetation 
preclude  the  use  of  heavier,  cart-based  systems.  Portability  can  improve  productivity  in  rough 
terrain.  The  smaller  sensor  head  of  the  MPV  (versus  the  TEMTADS  2x2  and  MetalMapper) 
means  that  it  is  able  to  access  areas  where  cart  and  vehicle  mounted  systems  are  too  large  to 
survey.  Sites  where  minimal  environmental  impact  is  required  are  well  suited  to  the  MPV. 
The  system  is  easily  packable  and  transportable; 

Multi-static,  Multi-channel  transmitter/receiver  design:  Five  receiver  cubes  record  three 
orthogonal  components  of  the  EM  field.  Multi-static  instruments  have  improved  dipole 
localization  relative  to  single,  coaxial  mono-static  instruments.  Multi-component,  multi-axis 
design  reduces  the  number  of  soundings  for  target  characterization  and  relaxes  positional 
accuracy.  Multiple  time  channels  over  a  wide  range  (up  to  25  ms)  provide  additional 
information  for  target  identification  and  scrap  rejection. 

-  Magnetic  soil  effects  can  be  reduced:  The  geometric  arrangement  of  receivers  and  the  wide¬ 
band  time  range  offer  potential  for  identifying  and  neutralizing  the  effect  of  magnetic  soil 
through  techniques  developed  in  SERDP  MM- 14 14  and  MM- 1573; 

-  Highly  stable  EMI  components:  Responses  are  directly  predictable  using  standard  EMI 
theory.  Field  tests  verified  that  MPV  components  has  minimal  measurement  temporal  drift; 

High  resolution  for  target  picking  in  areas  with  higher  target  density :  Having  several 
relatively  small  receivers  (8-cm  coils)  allows  localization  and  differentiation  of  individual 
anomalies  better  than  large  receivers  (e.g.,  EM61),  that  tend  to  “smear  out”  secondary  fields. 

Limitations  of  the  MPV  technology  include 

The  smaller  footprint  of  the  MPV  does  translate  into  a  lower  production  rate  for  dynamic 
surveying  in  open  areas  where  larger  footprint  systems  are  better  suited. 

The  MPV  does  not  sit  on  a  cart.  As  a  result,  operator  fatigue  is  a  greater  concern  as  the 
operator  can  be  expected  to  exert  more  effort  physically  transporting  the  system. 

-  Systems  with  a  larger  transmitter  loop  size  (e.g.  the  MetalMapper  “classic”)  are  better  able  to 
detect  and  classify  deeper  targets. 
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3.0  PERFORMANCE  OBJECTIVES 

This  project  includes  data  collection  in  dynamic  detection  and  cued  interrogation,  data 
analysis  and  user  feedback  for  evaluation  of  the  MPV  technology.  The  specific  objectives  for 
each  stage  are  detailed  in 

Table  1  and  2.  These  objectives  depend  on  the  intrinsic  data  quality  of  the  sensor,  the 
deployment  method  and  the  ensuing  data  analysis  and  interpretation. 


Table  1:  Performance  Objectives:  Data  Collection  Objectives 


Performance 

Objective 

Metric 

Data 

Required 

Success  Criteria 

Result 

Spatial 
coverage  in 
detection 
survey 

Extended 
footprint  coverage 

•  Mapped 
survey  data 

100%  coverage,  excluding 
obstacles  and  hazards 

99.43%  coverage 

Station 

spacing 

Distance  between 
soundings 

•  Sensor 
location 

98%  <  0.20  meter  (m); 

no  gaps  >  0.4  m  except 
obstacles  or  hazards 

99.99%  w/<0.20m 
spacing;  100% 
w/<0.40m  spacing 

Repeatability 
of  Instrument 
Verification 
Strip  (IVS) 
survey 

Amplitude  of  EM 
anomalies  and 
amplitude  of 
polarizabilities 

•  Twice-daily 
IVS  survey 
data 

Detection:  Within  a  factor 
of  2  on  detection  amplitude 
for  dynamic  IVS 
measurements  and  match 
metric  >  0.9  for  each  set  of 
three  inverted 
polarizabilities  for  cued 

IVS  measurements. 

117  of  120  dynamic 
amplitudes  within 
factor  of  2.  69  of  72 
cued  polarizabilities 
have  match  metric  > 
0.9 

Detection  of 
all  targets  of 
interest  (TOI) 

Percent  of  seeded 
items  detected 

•  Location  of 
seeded  items 

•  Anomaly  list 

100%  of  seeded  items 
detected  within  0.5  m  halo 

100%  of  21  seeds 
detected  within  0.5  m 

Cued 

interrogation 
of  anomalies 

Instrument 

position 

•  Cued  data 

100%  of  anomalies  where 
center  of  cued  pattern  is 
located  within  0.5  m  of 
anomaly  pick 

Data  were  acquired 
within  0.5  m  for 

100%  of  anomaly 
picks 

Production 

rate 

Acreage  and 
number  of  cued 
interrogations 

•  Log  of  field 
work 

Dynamic  mode:  0.7 
acre/day 

Cued  mode:  210  anomalies/ 
day  (7 -hour  survey  day) 

Dynamic  mode:  0.37 
acre/day 

Cued  mode:  278 
anomalies/day,  173 
unique  anomalies/day 
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Table  2:  Performance  Objectives:  Analysis  and  Classification  Objectives 


Performance 

Objective 

Metric 

Data 

Required 

Success  Criteria 

Result 

Maximize 

correct 

classification 

Number  of  TOI 
retained 

•  Ranked  dig 
list 

•  Scoring 
reports  by 

IDA 

Approach  correctly  identifies 
100%  of  TOI 

100%  correct 
classification 

Maximize 

correct 

classification 
of  non-TOI 

False  alarm  rate 
(FAR) 

•  Ranked  dig 
list 

•  Scoring 
reports  by 

IDA 

70%  reduction  of  clutter  digs 
at  100%  TOI  Pd  (or  stop  dig 
point  if  it  is  set  beyond  100% 
TOI  Pd) 

80.1%  of  non-TOI 
correctly  classified  as 
non-TOI 

Minimize 
number  of 
unclassifiable 
anomalies 

Number  of  “Can’t 
Analyze”  for  cued 
data  classification 

•  Ranked  dig 
list 

Less  than  5%  of  "Can't 
Analyze" 

0.3%  of  anomalies 
classified  as  “cannot 
analyze” 

Correct 
location  and 
depth  of  TOI 

Accuracy  of 
estimated  target 
parameters  for 
seed  items 

•  Results  of 
intrusive 
investigation 

•  Predicted 
location 

95%  TOI  with  AZ  <  0.10  m 

95%  TOI  with  AR  <  0.20  m 

100%  TOI  with  AZ  < 
0.1  m; 

88%  TOI  with  AR  < 

0.2  m 

3.1  OBJECTIVE:  SPATIAL  COVERAGE  FOR  DETECTION 

Dynamic  detection  survey  should  cover  a  maximum  of  the  area  of  interest  so  that  all 
detectable  targets  are  illuminated.  Targets  are  detectable  if  the  transmitted  field  is  sufficiently 
strong  to  reach  the  target  and  if  the  measured  target  response  is  sufficiently  strong  in  return  to 
exceed  a  given  threshold.  Simulations  and  analysis  of  field  data  suggest  that  there  is  negligible 
loss  of  detectability  when  a  target  is  located  0.1  m  off  to  the  side  of  the  MPV.  Given  the  0.5  m 
diameter  of  the  sensor  head,  one  can  assume  a  0.7  m  diameter  detection  footprint. 

3.1.1  Metric 

The  metric  is  the  spatial  coverage  of  the  MPV  detection  survey,  using  a  0.7  m  footprint, 
relative  to  the  surface  area  of  the  region  to  be  studied  in  dynamic  detection  mode. 

3.1.2  Data  requirements 

The  geographic  coordinates  for  the  perimeter  of  the  region  to  be  surveyed  and  the  MPV 
survey  track  is  utilized. 

3.1.3  Success  criteria  and  result 

Success  requires  100%  spatial  coverage  with  0.7  m  footprint  excluding  obstacles  and 
hazards.  Percent  coverage  for  each  house  is  shown  in  Table  3.  For  house  4  only  the  northwest 
portion  was  included  in  the  calculation.  The  southeast  part  of  house  4  and  all  of  house  6  were 
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excluded  due  to  lack  of  GPS  signal.  Overall  average  coverage  is  99.43%,  slightly  less  than  the 
specified  objective. 


Table  3:  Spatial  coverage  for  each  house. 


House 

Address 

%  Coverage 

1 

1745  Puako  Dr. 

99.77 

2 

1765  Puako  Dr. 

99.61 

3 

1853  Puako  Dr. 

99.78 

4  (NW) 

1877  Puako  Dr. 

99.14 

5 

1951  Puako  Dr. 

98.87 

6 

1971  Puako  Dr. 

NA 

3.2  OBJECTIVE:  STATION  SPACING  IN  DETECTION  MODE 

This  objective  is  meant  to  ensure  that  the  target  response  is  not  being  lost  as  a  result  of  gaps 
due  to  the  operator  moving  the  sensor  head  too  fast  or  due  to  missing  data. 

3.2.1  Metric 

The  metric  for  this  objective  is  the  distance  between  soundings  along  data  collection  lines. 

3.2.2  Data  requirements 

The  sensor  head  location  is  derived  from  GPS  and  Attitude  and  Heading  Reference  Sensor 
(AHRS)  measurements  which  will  be  used  to  compute  this  metric  and  map  the  EMI  data. 

3.2.3  Success  criteria  and  result 

Success  requires  that  98%  of  the  data  points  have  at  most  0.20  m  spacing  along  data 
collection  lines  and  100%  have  less  than  0.40  m  spacing.  Station  spacing  results  for  each  house 
are  shown  in  Table  4.  Both  objectives  were  met. 

Table  4:  Percentage  of  stations  spacing  less  than  0.4  m  and  0.2  m  for  each  house. 


House 

Address 

%  <=  0.4  m 

%  <=  0.2  m 

1 

1745  Puako  Dr. 

100.00 

99.98 

2 

1765  Puako  Dr. 

100.00 

99.99 

3 

1853  Puako  Dr. 

100.00 

100.00 

4  (NW) 

1877  Puako  Dr. 

100.00 

100.00 

5 

1951  Puako  Dr. 

100.00 

99.98 

6 

1971  Puako  Dr. 

NA 

NA 
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3.3  OBJECTIVE:  REPEATABILITY  OF  INSTRUMENT  VERIFICATION  TESTS 

The  reliability  of  survey  data  depends  on  the  consistent  performance  of  the  survey 
equipment.  This  objective  concerns  twice-daily  verification  on  a  test  strip  where  metallic  targets 
will  be  buried  to  confirm  the  sensor  system  performance.  The  IVS  is  surveyed  in  detection  mode 
during  the  detection  survey.  The  IVS  targets  are  surveyed  in  cued  interrogation  during  the  entire 
demonstration. 

Unfortunately  field  crews  were  directed  to  move  the  IVS  location  multiple  times,  hampering 
the  repeatability.  The  IVS  had  been  placed  on  a  shoulder  of  a  public  home  so  as  to  not 
inconvenience  any  one  homeowner  with  twice  daily  IVS  tests  for  the  duration  of  the  survey 
work.  Initially  the  buried  IVS  items  were  required  to  be  removed  from  subsurface  and  placed  on 
the  surface  then  subsequently  had  to  be  moved  to  a  co-operative  homeowner’s  property  for 
completion  of  the  surveys. 

3.3.1  Metrics 

The  metric  for  detection  relates  to  the  amplitude  of  the  maximum  target  response,  defined  as 
the  norm  of  the  total  field  on  each  receiver  cube  for  the  0.5  msec  time  channel.  The  metric  for 
cued  interrogation  is  the  target  size,  here  defined  as  the  norm  of  the  polarizability  components 
also  for  the  0.5  msec  time  channel. 

3.3.2  Data  requirements 

IVS  data  are  recorded  for  both  detection  and  cued  survey  modes.  A  detection  map  is  built 
and  the  detection  amplitude  is  computed  for  each  target.  For  the  cued  survey  the  data  are 
inverted  and  the  stability  of  the  recovered  target  parameters  is  verified. 

3.3.3  Success  criteria  and  result 

The  detection  requirement  is  a  factor  2  on  the  target  response  for  dynamic  IVS  measurements 
and  a  match  metric  to  median  polarizabilities  of  >  0.9  for  each  set  of  three  inverted 
polarizabilities  for  cued  IVS  measurements. 

Results  for  30  sets  of  dynamic  IVS  measurements  are  shown  in  Table  5.  All  but  three 
measurements  from  two  different  visits  to  the  IVS  (highlighted  in  yellow)  successfully  met  the 
requirement. 
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Table  5:  Amplitude  of  dynamic  IYS  measurements  and  value  relative  to  reference  amplitudes.  “Amp.  ratio” 
is  defined  as  max(Ai,Ri)/min(Ai,Ri),  where  Ai  is  the  measured  amplitude  for  the  i-th  IYS  item,  and  Ri  is  the 
reference  amplitude  for  the  i-th  IYS  item  (see  Table  6).  Success  is  achieved  when  the  amplitude  ratio  is  <  2. 


N 

Date/ID 

RMS 
amp  1 

RMS 
amp  2 

RMS 
amp  3 

RMS 
amp  4 

Amp. 
ratio  1 

Amp. 
ratio  2 

Amp. 
ratio  3 

Amp. 
ratio  4 

1 

151203_AM_05 

18.01 

47.28 

31.56 

31.48 

1.47 

1.00 

1.02 

1.59 

2 

151203 AM 06 

19.22 

38.68 

28.38 

53.68 

1.38 

1.22 

1.13 

1.07 

3 

151203_PM_11 

36.65 

40.3 

32.15 

35.48 

1.39 

1.17 

1.00 

1.41 

4 

151203_PM_12 

18.57 

65.76 

24.27 

68.57 

1.42 

1.39 

1.32 

1.37 

5 

151204_PM_19 

30.26 

42.84 

27 

43.51 

1.14 

1.10 

1.19 

1.15 

6 

151204_PM_20 

22.7 

65.9 

35.19 

68.14 

1.17 

1.39 

1.10 

1.36 

7 

151207_AM_25 

29.42 

36.32 

32.66 

50.08 

1.11 

1.30 

1.02 

1.00 

8 

151207_AM_26 

26.75 

57.87 

44.45 

53.01 

1.01 

1.22 

1.39 

1.06 

9 

151207_PM_37 

29.02 

48.03 

25.17 

37.84 

1.10 

1.02 

1.27 

1.32 

10 

151207_PM_38 

22.1 

53.64 

35.54 

60.56 

1.20 

1.13 

1.11 

1.21 

11 

151208_AM_45 

26.45 

27.16 

22.37 

22.43 

1.00 

1.74 

1.43 

2.23 

12 

151208_AM_46 

15.31 

29.42 

32.06 

53.17 

1.73 

1.61 

1.00 

1.06 

13 

151208_PM_53 

28.72 

53.23 

37.09 

44.01 

1.09 

1.13 

1.16 

1.14 

14 

151208_PM_54 

24.97 

41.28 

46.35 

62.44 

1.06 

1.15 

1.45 

1.25 

15 

151209_AM_61 

30.75 

32.2 

29.19 

28.27 

1.16 

1.47 

1.10 

1.77 

16 

151209_AM_62 

22.01 

33.91 

31.95 

63.42 

1.20 

1.39 

1.00 

1.27 

17 

151209 PM 69 

29.7 

28.31 

29.59 

31.42 

1.12 

1.67 

1.08 

1.59 

18 

151209_PM_70 

23.76 

45.25 

29.53 

51.09 

1.11 

1.04 

1.09 

1.02 

19 

151210_AM_75 

21.64 

19.96 

23.87 

21.47 

1.22 

2.37 

1.34 

2.33 

20 

151210_AM_76 

14.09 

31.94 

20.62 

35.42 

1.88 

1.48 

1.55 

1.41 

21 

151210_PM_93 

40.01 

87.85 

35.23 

48.66 

1.51 

1.86 

1.10 

1.03 

22 

151210_PM_94 

32.91 

85.37 

39.43 

79.09 

1.24 

1.81 

1.23 

1.58 

23 

151214_AM_01 

30.65 

64.53 

36.13 

44.26 

1.16 

1.36 

1.13 

1.13 

24 

151214_AM_02 

23.13 

50.54 

27.13 

61.7 

1.14 

1.07 

1.18 

1.23 

25 

151214_PM_08 

36.61 

87.92 

43.05 

47.12 

1.38 

1.86 

1.34 

1.06 

26 

151214_PM_09 

25.51 

74.49 

41.26 

67.25 

1.04 

1.58 

1.29 

1.34 

27 

151215_AM_14 

33.62 

49.4 

33.94 

53.32 

1.27 

1.04 

1.06 

1.06 

28 

151215_AM_15 

22.55 

53.84 

29.14 

50.48 

1.17 

1.14 

1.10 

1.01 

29 

151215_PM_20 

30.13 

46 

32.28 

34.42 

1.14 

1.03 

1.01 

1.45 

30 

151215_PM_21 

27.02 

49.62 

31 

27.63 

1.02 

1.05 

1.03 

1.81 

Table  6:  Reference  amplitudes  for  dynamic  IYS  measurements. 


Ref  Amp  1 

Ref  Amp  2 

Ref  Amp  3 

Ref  Amp  4 

26.45 

47.28 

32.06 

50.08 
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Results  for  18  sets  of  cued  I  VS  measurements  are  shown  in  Table  7.  All  but  three 
measurements  from  two  different  visits  to  the  IVS  (highlighted  in  yellow)  successfully  met  the 
requirement. 


Table  7:  Polarizability  match  metrics  for  18  sets  of  cued  IVS  measurements.  Success  is  achieved  when  the 

match  metric  is  >  0.9. 


N 

Date/ID 

Match 

1 

Match 

2 

Match 

3 

Match 

4 

1 

151203 

0.91 

0.98 

1.00 

0.99 

2 

151204 AM 

0.88 

0.91 

1.00 

0.99 

3 

151204 PM 

0.95 

0.96 

1.00 

0.96 

4 

151207 AM 

0.94 

0.99 

0.99 

1.00 

5 

151207 PM 

0.92 

0.96 

0.99 

0.98 

6 

151208 AM 

0.98 

0.94 

0.99 

1.00 

7 

151208 PM 

0.99 

0.98 

1.00 

1.00 

8 

151209 AM 

0.98 

0.98 

1.00 

1.00 

9 

151209 PM 

0.95 

0.96 

0.99 

0.99 

10 

151210 AM 

0.99 

0.98 

1.00 

1.00 

11 

151210 PM reposn 

0.99 

0.99 

1.00 

0.99 

12 

151214 AM 

0.98 

1.00 

0.99 

0.99 

13 

151214 PM 

0.99 

0.98 

1.00 

0.99 

14 

151215 AM 

0.99 

0.97 

1.00 

0.98 

15 

151215 PM 

0.92 

0.98 

1.00 

1.00 

16 

151216 AM 

0.96 

0.97 

0.97 

0.98 

17 

151217 AM 

0.91 

0.95 

0.95 

0.96 

18 

151217_PM_wAMbg 

0.87 

0.87 

0.98 

0.98 

3.4  OBJECTIVE:  DETECTION  OF  ALL  TARGETS  OF  INTEREST 

Quality  data  should  lead  to  high  probability  of  detecting  all  TOI  at  the  site. 

3.4.1  Metric 

The  metric  for  this  objective  is  the  percentage  of  seed  items  that  are  detected  using  the 
specified  anomaly  detection  threshold. 

3.4.2  Data  requirements 

The  demonstrator  normally  submits  a  detection  list  to  the  Program  Office  for  evaluation.  In 
this  case  detection  lists  were  sent  to  Parsons  QC  Geophysicist  who  verified  detection  of  QC 
seeds. 
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3.4.3  Success  criteria  and  result 

The  objective  will  be  considered  to  be  met  if  100%  of  the  seeded  items  are  detected  within  a 
0.5-m  halo. 

The  Demonstration  Plan  (BTG  and  Parsons,  2015)  predicted  a  maximum  depth  of  detection 
for  a  small  ISO  to  be  25cm  based  on  background  noise  levels  measured  from  the  previous 
Waikoloa  MPV  demonstration.  The  project  detection  threshold  was  therefore  established  to 
detect  a  small  ISO  buried  at  25cm.  All  twenty-one  seeds  buried  within  the  design  specification 
interval  between  surface  and  25cm  and  surveyed  with  the  MPV  were  detected. 

Four  other  small  ISO  seeds  (WK-31,  -33,  -35  and  -36)  were  buried  deeper  than  25cm  and 
were  detected  and  correctly  classified  using  the  project's  detection  and  classification  parameters. 
Because  these  were  buried  to  empirically  assess  performance  at  depths  greater  than  those 
determined  for  the  site  as  a  whole,  they  are  excluded  from  the  total  of  twenty-one  reported 
above,  which  were  buried  within  the  design  specification  of  25cm.  One  seed,  WK-34,  a 
horizontal  small  ISO,  was  buried  at  28cm  (also  beyond  the  design  specification)  and  was  not 
detected. 

Some  seeds  were  not  covered  because  homeowners  specifically  requested  that  garden  and 
landscaping  areas  not  be  surveyed.  Other  seeds  were  considered  not  covered  by  the  dynamic 
surveys,  mostly  because  a  consistent  GPS  signal  could  not  be  maintained  while  the  instrument 
was  moving.  These  were  cases  of  GPS  satellites  being  obscured  by  tree  canopy  or  proximity  to 
buildings. 

3.5  OBJECTIVE:  CUED  INTERROGATION  OF  ANOMALIES 

The  reliability  of  cued  data  depends  on  acceptable  instrument  positioning  during  data 
collection  in  relation  to  the  actual  anomaly  location. 

3.5.1  Metric 

The  metric  for  this  objective  is  the  percentage  of  anomaly  peaks  that  are  located  within  the 
acceptable  distance  to  the  center  of  the  cued  interrogation  survey  of  each  anomaly. 

3.5.2  Data  requirements 

The  demonstrator  records  the  location  of  their  instrument  for  each  cued  anomaly  interrogated 
and  verifies  that  the  anomaly  is  covered  by  the  survey  pattern.  Verification  is  done  while  still  on 
site  so  that  anomalies  can  be  re-acquired  if  needed. 

3.5.3  Success  criteria  and  result 

The  objective  will  be  considered  to  be  met  if  the  center  of  the  instrument  pattern  is  positioned 
within  the  0.5  m  distance  of  the  actual  anomaly  location  for  100%  of  the  cued  anomalies.  Result: 
data  were  acquired  within  0.5  m  for  100%  of  all  picked  anomalies  in  areas  that  had  GPS 
coverage. 
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3.6  OBJECTIVE:  PRODUCTION  RATE 

This  objective  addresses  the  time  taken  for  data  collection. 

3.6.1  Metric 

The  metrics  for  this  objective  are  the  mean  daily  survey  rates  in  terms  of  acreage  for  dynamic 
survey  and  number  of  targets  for  cued  interrogations. 

3.6.2  Data  requirements 

The  acreage  and  number  of  surveyed  anomalies  were  recorded  on  every  day. 

3.6.3  Success  criteria  and  result 

The  objective  will  be  met  if  the  mean  daily  survey  rates  are  at  least  0.7  acre  and  210 
anomalies  based  on  7  hours  of  effective  data  collection  (excluding  weather  interruptions  and 
other  long  breaks). 

Total  area  surveyed  in  dynamic  mode  for  houses  1  to  5  combined  is  0.72  acres.  Time  to 
survey  these  houses  was  13.35  hours  (Table  8),  giving  a  mean  daily  survey  rate  of  0.38  acre/day, 
assuming  a  7-hour  work  day.  This  is  below  the  objective  of  0.7  acre/day  which  was  a  number  set 
based  on  previous  surveys  in  obstacle  free,  open  sky  conditions.  Surveying  in  complicated 
residential  environments  results  in  lower  production  rates.  House  6  was  omitted  from  this 
calclation  due  to  lack  of  GPS  coverage  resulting  in  uncertainty  in  the  area  surveyed  for  this 
house. 

Table  8:  Hours  of  dynamic  surveying  for  each  house,  by  day. 


Day 

House 

Hours  dyn  surveying 

12/3/2015 

1 

0.10 

12/4/2015 

1 

1.67 

12/7/2015 

1 

1.49 

12/7/2015 

2 

3.29 

12/8/2015 

2 

0.78 

12/8/2015 

3 

1.60 

12/8/2015 

4 

1.00 

12/9/2015 

4 

0.52 

12/9/2015 

5 

2.91 

Total 

13.35 

The  overall  average  number  of  cued  acquisitions  per  7-hour  day  was  278  (Table  9)  which  is 
above  the  project  established  threshold  of  210  anomalies  per  day.  The  average  number  of  unique 
anomalies  cued  per  7-hour  day  was  173. 
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Table  9:  Cued  anomaly  production  rates  for  each  house,  by  day.  Unique  anom  is  the  unique  number  of 
anomalies  cued  (i.e.,  not  counting  recollects).  Daily  rates  are  based  on  a  7-hour  day. 


Day 

House 

Hours 

Anomalies 

Unique  anom 

Anom/day 

Unique  anom/day 

12/10/2015 

1 

8.62 

204 

106 

166 

86 

12/14/2015 

1 

5.80 

232 

149 

280 

180 

12/14/2015 

2 

2.42 

168 

105 

486 

304 

12/15/2015 

2 

3.84 

137 

89 

250 

162 

12/15/2015 

3 

1.47 

77 

51 

368 

244 

12/15/2015 

4 

1.13 

68 

47 

421 

291 

12/16/2015 

5 

2.37 

143 

92 

423 

272 

12/16/2015 

6 

2.49 

86 

56 

242 

158 

12/17/2015 

1 

0.62 

14 

7 

158 

79 

12/17/2015 

2 

0.34 

2 

2 

42 

42 

12/17/2015 

4 

0.24 

22 

13 

644 

381 

12/17/2015 

5 

0.41 

13 

9 

224 

155 

12/17/2015 

6 

0.14 

19 

14 

971 

715 

Total 

29.87 

1185 

740 

278 

173 

3.7  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  TOI 

This  is  one  of  the  two  primary  measures  of  the  effectiveness  of  the  classification  approach. 
By  collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter  estimation  and 
classification  algorithms,  targets  will  be  classified  with  high  efficiency.  This  objective  concerns 
the  component  of  the  classification  problem  that  involves  correct  classification  of  TOI. 

3.7.1  Metric 

The  metric  for  this  objective  is  the  number  of  items  on  the  anomaly  list  for  a  particular  sensor 
that  can  be  correctly  classified  as  TOI  by  each  classification  approach. 

3.7.2  Data  requirements 

BTG  will  prepare  a  ranked  anomaly  list  for  the  targets  on  the  sensor  anomaly  list.  Parsons 
QC  Geophysicist  ensured  that  all  QC  seeds  surveyed  were  detected  and  correctly  classified  on 
submitted  dig  lists. 

3.7.3  Success  criteria  and  result 

The  objective  will  be  considered  to  be  met  if  100%  of  the  TOI  are  correctly  labeled  as  TOI 
on  the  ranked  anomaly  list. 

Individual  dig  lists  were  submitted  for  each  of  the  six  houses.  All  TOI  surveyed  at  all  houses 
were  correctly  labeled  as  high-confidence  TOI. 
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3.8  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOI 

This  is  the  second  of  the  two  primary  measures  of  the  effectiveness  of  the  classification 
approach.  By  collecting  high-quality  data  and  analyzing  those  data  with  advanced  parameter 
estimation  and  classification  algorithms,  targets  will  be  classified  with  high  efficiency.  This 
objective  concerns  the  component  of  the  classification  problem  that  involves  false  alarm 
reduction. 

3.8.1  Metric 

The  metric  for  this  objective  is  the  number  of  items  on  the  sensor  dig  list  that  can  be  correctly 
classified  as  non-TOI  by  each  classification  approach. 

3.8.2  Data  requirements 

BTG  will  prepare  a  ranked  anomaly  list  for  the  targets  on  the  sensor  anomaly  list.  Parsons 
personnel  will  provide  confirmation  that  all  QC  seeds  are  correctly  classified  as  TOI. 

3.8.3  Success  criteria  and  result 

The  objective  will  be  considered  to  be  met  if  more  than  70%  of  the  non-TOI  items  can  be 
correctly  labeled  as  non-TOI  while  retaining  100%  of  the  TOI  on  the  dig  list.  If  the  analyst 
chosen  stop  dig  point  occurs  past  the  point  on  the  dig  list  where  100%  of  TOI  are  identified,  the 
extra  digs  beyond  the  identification  of  100%  of  the  TOI  and  the  analyst’s  stop  dig  point  will  be 
factored  into  the  70%  reduction  of  non-TOI  items  objective. 

As  shown  inTable  9  of  the  604  non-TOI  encountered  at  the  five  properties,  80.1%  of  the  non- 
TOI  were  correctly  labeled.  Hence  the  demonstration  objective  of  better  than  70%  correct 
classification  was  met. 


Table  10:  Data  for  calculating  the  %  on  non-TOI  correctly  labeled  as  non-TOI.  Note  that  for  house  1 
complete  ground  truth  is  not  available.  The  estimate  for  house  1  assumes  that  all  un-dug  anomalies  are  non- 

TOI. 


House 

# 

Anomalies 

Stop 

dig 

#  Non- 

TOI 

#  Digs  after 
stop  dig 

%  Non-TOI 
correctly  labeled 

Comment 

1 

231 

50 

226 

181 

80.1 

Assuming  170  un-dug 
targets  are  non-TOI 

2 

194 

41 

187 

153 

81.8 

3 

51 

17 

45 

34 

75.6 

4 

60 

24 

57 

37 

64.9 

5 

96 

17 

89 

79 

88.8 

Total 

632 

604 

484 

80.1 
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3.9  OBJECTIVE:  MINIMUM  NUMBER  OF  UN  CLASSIFIABLE  ANOMALIES 

Anomalies  for  which  reliable  parameters  cannot  be  estimated  cannot  be  classified  by  the 
classifier.  These  anomalies  must  be  placed  in  the  dig  category  and  reduce  the  effectiveness  of  the 
classification  process. 

3.9.1  Metric 

The  metric  is  the  number  of  anomalies  that  cannot  be  analyzed  by  our  method. 

3.9.2  Data  requirements 

The  submitted  dig  list  specifies  those  anomalies  for  which  parameters  could  not  be  reliably 
estimated. 

3.9.3  Success  criteria  and  results 

The  objective  will  be  met  if  less  than  5%  of  the  cued  anomalies  cannot  be  analyzed. 

Two  anomalies  (0.3%  of  the  632  anomalies  at  houses  1  to  5)  were  classified  as  “cannot 
analyze”:  1139  at  house  1;  and  4048  at  house  4  because  cued  data  were  not  collected  over  these 
anomalies.  The  objective  of  <  5%  was  achieved. 

3.10  OBJECTIVE:  CORRECT  ESTIMATION  OF  LOCATION  AND  DEPTH 

Correct  target  classification  relies  on  the  capability  to  extract  valid  target  parameters. 
Accurate  TOI  location  is  also  important  for  safe  and  efficient  site  remediation. 

3.10.1  Metric 

The  metric  is  the  difference  between  observed  and  predicted  depth  and  geographic  location. 

3.10.2  Data  requirements 

Target  location  and  depth  are  recorded  and  compared  to  ground-truth  validation 
measurements.  This  objective  requires  accurate  ground  truth  documentation. 

3.10.3  Success  criteria  and  result 

Depth  should  be  predicted  within  0.10  m  and  geographic  location  within  0.20  m  for  95%  of 
TOI. 

For  the  25  TOI  with  ground  truth  information,  22  of  25  (88%)  of  the  predicted  locations  were 
with  0.2  m  of  the  ground  truth  location.  The  predicted  depths  of  all  25  TOI  were  within  0.1  m  of 
the  ground  truth  depths.  The  objective  for  location  error  (95%  with  0.2  m)  was  not  achieved.  The 
objective  for  depth  error  was  achieved. 
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4.0  SITE  DESCRIPTION 

A.l  Site  map 

The  Former  Waikoloa  Maneuver  Area  is  located  on  the  northwest  side  of  the  Big  Island  of 
Hawaii.  The  demonstration  will  be  conducted  in  the  northeast  area  of  Sector  17C  within  the  Former 
Waikoloa  Maneuver  Area.  The  demonstration  area  is  within  Sector  17C  that  is  included  in  the 
current  remedial  investigation  (RI)  and  is  approximately  1,020  acres  located  in  the  coastal  portion  of 
WMA  west  of  Queen  Ka‘ahumanu  Highway.  The  six  residential  properties  surveyed  in  the 
demonstration  are  shown  in  Figure  4. 


Figure  4-1 

Demonstration  area 
Former  Waikoloa  Maneuver  Area 


Hawaii  County,  Hawaii 


Legend 

□  Parcels  -  Hawaii  County 
~1  Sector  Boundary 
|  Data  Collection  Area 


500 
■  Feet 


April,  2016 


Figure  4:  Survey  area  map  (from  Parsons,  2016).  Six  residential  properties  were  selected  for  the  study. 

A.2  Munitions  contamination 


Suspected  munitions  included: 

•  60-mm  and  80-mm  high  explosive  mortars 

•  75-mm,  105-mm,  and  155-mm  projectiles 

•  2.36-inch  rocket  propelled  anti-tank  rounds 

•  US  MK II  hand  grenades 

•  Rockets 

•  Ml  anti-tank  land  mines 

•  Japanese  ordnance 
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5.0  TEST  DESIGN 

5.1  EXPERIMENTAL  DESIGN 

There  were  two  main  objectives  to  this  project: 

•  Validate  the  performance  of  the  MPV  in  a  semi-urban  area  containing  magnetic  soils. 

•  Develop  enough  data  to  evaluate  the  use  of  the  MPV  as  part  of  a  remedial  alternative  in  the 
feasibility  study  to  be  prepared  for  WMA  Sector  17. 

The  key  components  of  this  method  were  seeding  the  site  to  determine  the  effectiveness  of  the 
MPV;  collecting  dynamic  MPV  data  across  the  site  and  selecting  anomalous  areas  in  that  data; 
collecting  cued  MPV  data  over  identified  anomalies;  analyzing  the  advanced  sensor  using  physics- 
based  models  to  extract  target  parameters  such  as  size,  shape,  and  materials  properties;  using  those 
parameters  to  construct  a  ranked  anomaly  list;  and  intrusively  investigating  the  targets.  This  report 
covers  only  the  data  collection,  processing  and  classification  components  of  the  project;  Parson’s 
demonstration  report  (2016)  discusses  the  seeding  and  intrusive  investigation  portions  of  the  project. 

5.2  SYSTEM  SPECIFICATION 

5.2.1  Data  acquisition 

For  cued  interrogation  mode  the  system  is  set  for  25  ms  excitation  and  25  ms  recording  of 
EMI  transients  (100  ms  per  cycle).  This  is  accomplished  by  setting  the  acquisition  parameters  to 
0.9  seconds  (sec)  data  blocks  and  9  repeats.  Station  time  is  set  to  6.3  sec  by  stacking  7  data 
blocks  (effectively  9  x  7  =  63  cycles  are  averaged).  Digital  receivers  use  a  4  microsecond 
sampling  rate.  The  data  are  recorded  with  133  logarithmically-spaced  time  gates  (5%  gate  width) 
from  0-25  ms.  Dynamic  survey  is  set  with  2.7  ms  time  decay  and  short  0.1  second  data  block  to 
reduce  smearing  of  the  signal  by  sensor  motion. 

5.2.2  Positioning  and  navigation 

The  dynamic  area  was  expected  to  be  open  sky  and  with  positioning  based  on  RTK  GPS.  The 
GPS  used  is  a  Trimble  R8  that  is  mounted  on  the  opposite  end  of  the  MPV  handling  boom.  The 
GPS  is  also  used  to  locate  pre-programmed  flag  locations  for  cued  measurements.  An  XSens 
MTi  orientation  sensor  is  mounted  near  the  GPS  to  predict  the  sensor  head  location  relative  to 
the  GPS. 

5.3  CALIBRATION  ACTIVITIES 

Calibration  is  designed  to  verify  correct  sensor  operation  and  calibrate  the  recorded  sensor 
response  over  known  targets.  A  sample  set  of  the  expected  targets  were  calibrated  with  test  pit 
measurements.  Each  sample  was  successively  placed  inside  a  clutter-free  training  pit  and 
surveyed  in  cued  interrogation  mode.  A  range  of  different  orientations  and  depths  per  target  were 
acquired  to  train  feature  extraction  and  classification  methods.  Data  were  inverted  on  that  day  to 
verify  the  stability  of  the  recovered  target  parameters. 

The  IVS  was  surveyed  for  calibration  and  sensor  verification  in  dynamic  detection  and  cued 
interrogation  modes.  The  IVS  was  surveyed  multiple  times  for  training  in  both  modes,  and  twice 
daily  in  the  collection  mode  of  the  day  for  verification.  The  detection  data  were  analyzed  to 
verify  spatial  coverage  and  the  stability  of  the  EMI  responses,  thus  providing  an  indirect  check 
on  the  data  collection  procedure  and  on  the  sensor  components.  The  amplitude  of  the  target 
responses  were  also  used  for  calibration  against  the  detection  threshold.  The  dynamic  data  were 
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inverted  to  recover  the  dynamic  polarizabilities  of  the  buried  targets.  These  were  used  for 
detection  simulations  and  classification  of  dynamic  data.  The  cued  data  were  also  inverted  to 
recover  the  static  polarizabilities,  verify  their  stability,  and  provide  training  data  for 
classification. 

Geologic  background  measurements  in  cued  mode  were  acquired  at  the  IVS  over  the  location 
determined  to  be  free  of  metal  sources  during  construction  of  the  IVS.  At  the  six  residential 
properties,  “quiet”  areas  were  identified  for  potential  background  locations  based  on  the  dynamic 
survey  data.  Background  location  measurements  were  analyzed  to  quantify  the  spatial  and 
temporal  variability  in  background  noise  due  to  soil  magnetization. 

5.4  DATA  COLLECTION  PROCEDURES 
5.4.1  Detection  survey 

Detection  survey  is  performed  by  walking  along  pre-defined  survey  lines.  The  sensor  has  an 
effective  detection  footprint  of  0.7  m  although  the  largest  cube  separation  is  0.47  m  (between 
cube  centers).  To  ensure  full  coverage  the  detection  survey  was  run  at  0.5-m  line  spacing. 
Practically,  field  operators  laid  survey  ropes  on  the  ground  at  1-m  spacing  and  used  the  lines  to 
guide  the  exterior  side  of  the  MPV  sensor  head  (Figure  5). 


Figure  5:  MPV  operator  surveying  in  dynamic  mode  following  rope  lanes.  Operator  follows  straight  lines. 
5.4.2  Cued  interrogation 

In  previous  demonstrations,  cued  MPV3D  measurements  have  been  collected  around  the 
marked  target  location  (ground  paint  or  flag)  or  by  navigating  to  a  precise  location  using  the 
navigation  capabilities  of  the  MPV.  For  these  surveys,  MPV  navigation  was  used  to  return  to  the 
cued  measurement  locations  selected  from  the  dynamic  MPV  surveys.  For  the  MPV3D,  the 
sensor  was  placed  at  the  picked  location.  The  data  were  immediately  inverted.  If  the  recovered 
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target  location  was  offset  by  more  than  0.2  m,  a  second  measurement  was  acquired  at  the  new 
predicted  location. 


Figure  6:  MPV3D  cued  interrogation.  The  first  sounding  takes  place  at  the  picked  location.  The 
data  are  inverted  and  a  second  measurement  might  be  taken  in  the  case  of  large  predicted  offset. 

5.4.3  Positioning  and  navigation 

RTK  GPS  was  used  to  locate  the  sensor  for  mapping,  and  for  re-acquiring  targets  for  cued 
interrogation.  The  GPS  data  were  ingested  by  the  DAQ  to  indicate,  in  real-time,  the  sensor 
location.  Detection  lines  were  laid  on  the  ground  as  rope  lanes  and  observed  on  the  control 
display  in  real  time  by  the  operator  to  track  the  real-time  spatial  coverage.  Cued  anomalies  were 
preloaded  so  that  the  GPS  could  be  used  for  navigation  to  these  anomalies. 

5.4.4  Quality  checks 

A  general  check  on  proper  operation  was  verified  every  time  the  instruments  are  powered  on. 
The  positioning  systems  were  checked  by  waving  the  MPV  head  and  verifying  on  the  screen 
display  that  the  reported  position  and  orientation  numbers  as  well  as  the  location  map  were  being 
updated  and  vary  as  predicted.  The  EMI  elements  were  checked  by  acquiring  data  in  dynamic  or 
static  mode,  depending  on  the  stage  of  the  project.  The  operator  verified  that  the  "dancing 
arrows"  display  was  updated  in  response  to  variations  in  the  EM  environment,  that  signals  were 
appearing  in  the  signal  time-decay  display  (Figure  7)  and  that  a  file  was  being  written. 
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Battery  change  was  accompanied  with  a  basic  system  check  although  the  cDAQ  was  not 
necessarily  shut  down  (hot-swap  of  the  batteries).  A  background  soil  measurement  and  an  in-air 
measurement  were  acquired  in  the  current  survey  mode  (dynamic  or  static)  before  and  after  the 
battery  swap.  The  operator  checked  the  display  for  anomalous  behavior.  The  data  were  later 
examined  in  post  processing  to  identify  and  correct  any  sensor  drift.  In  addition,  background 
measurements  for  the  soil  response,  with  the  sensor  on  the  ground,  and  the  in-air  response  were 
frequently  acquired.  The  former  test  was  to  document  the  variability  in  the  soil  response  and 
ensure  that  the  most  relevant  background  was  applied  -  a  magnetic  soil  response  would  mostly 
affect  the  late  time  data  and  may  appear  similar  to  the  presence  of  a  large  deep  target.  The  in-air 
measurement  was  designed  to  capture  the  intrinsic  sensor  response  as  a  function  of  the  battery 
power,  which  varies  as  the  battery  drains  out.  That  response  is  particularly  important  at  early 
time,  during  the  0.3  msec  after  the  transmitter  turns  off,  when  a  large  inductive  response  is 
observed  in  the  Z-component  receivers  due  to  their  coupling  with  the  Z-axis  transmitter  (the  so- 
called  "transmitter  ringing"  effect). 

Dynamic  acquisition  was  continuously  monitored  by  verifying  that  the  sensor  location  map, 
the  positioning  data  table  and  the  dancing  arrows  were  being  updated.  In  particular,  the  map 
would  show  a  sensor  track  that  covered  the  survey  line  without  any  gaps;  a  pop-up  window 
would  appear  if  data  errors  were  encountered;  the  dancing  arrows  should  move  around  in 
response  to  changes  in  the  sensor  clearance  or  the  presence  of  metallic  objects.  The  second 
operator  was  also  involved  in  quality  control  by  verifying  that  the  front  operator  was  keeping  the 
sensor  head  close  to  the  ground,  covering  the  entire  line  and  keeping  a  somewhat  uniform  pace. 


Figure  7:  Typical  target  response  when  the  MPV  head  is  placed  directly  above  a  buried  target. 

The  Z-component  data  shows  that  target  is  closest  to  the  center  cube  (#3)  and  equally  distant  from  lateral 
cubes  2  and  4,  while  signal  in  cube  5  resembles  background.  The  Y  data  confirm  that  target  is  buried  between 
front  and  back  cubes  (1, 5)  and  X  data  confirm  that  target  is  located  between  side  cubes  2  and  4. 


Cued  interrogation  followed  a  specific  protocol.  Each  sounding  was  displayed  immediately 
after  acquisition  to  verify  proper  sensor  operation  and  correct  characterization  of  the  buried 
target.  The  operator  verified  that  receivers  were  properly  operating  by  examining  data  decay 
curves  (Figure  7)  and  the  "dancing  arrows"  display.  Any  abnormal  sounding  were  deleted, 
reacquired  at  the  same  location  and  flagged  in  field  notes  to  differentiate  it  from  acceptable 
soundings.  If  receiver  failure  occurred  the  survey  would  be  stopped  until  a  solution  was  found. 
Correct  characterization  of  an  anomaly  followed  a  series  of  steps: 

•  The  first  sounding  required  particular  attention  to  verify  that  the  signal  source  originated 
right  below  the  marked  location.  There  can  be  an  offset  between  the  picked  anomaly 
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location,  where  the  MPY  would  be  placed,  and  the  apparent  target  location  that  is  predicted 
by  the  current  MPV  data  -  this  can  arise  from  positional  error,  choice  of  a  target  picking 
algorithm,  or  the  presence  of  multiple  targets.  In  case  of  large  apparent  offset  the  operator 
was  expected  to  try  and  interpret  the  cued  interrogation  data  to  locate  the  signal  source  and 
acquire  additional  soundings  if  necessary; 

•  Anomaly  coverage  was  verified  by  ensuring  that  the  furthest  receiver  measured  background. 
If  residual  signal  from  the  target  remained,  then  additional  soundings  were  collected  to 
ensure  full  coverage  of  the  anomaly  spatial  decay.  For  instance,  if  the  MPV  front  receivers 
showed  above-background  signal  when  the  MPV  was  placed  in  position  2  of  Figure  6,  then  a 
sounding  was  to  be  collected  North  of  the  middle  of  positions  2-3.  If  a  nearby,  interfering 
target  was  detected  while  being  un-flagged  for  cued  interrogation,  then  supplementary 
soundings  are  acquired  to  improve  characterization  of  the  two  sources. 

Off-site  data  quality  review  was  performed  on  a  daily  basis  by  importing  dynamic  and  cued 
data.  The  geophysicist  loaded  up  the  data  to  verify  that  positioning  and  EMI  sensors  were 
properly  functioning,  that  noise  levels  were  normal,  GPS  yielded  realistic  positions  and  that 
spatial  coverage  was  sufficient.  In  particular  the  analyst  checked  for  gaps  in  the  dynamic 
detection  map,  and  verified  that  anomalies  were  fully  covered  in  cued  mode.  If  problems 
occurred  then  causes  were  investigated  and  the  affected  survey  lines  or  anomalies  were 
resurveyed  if  necessary. 

The  last  check  was  verification  that  all  targets  had  been  visited.  In  the  open  field  we  kept 
track  of  all  anomalies  by  having  pre-programmed  their  GPS  coordinates  and  displaying  their 
location  on  the  sensor  display  map.  Each  visited  target  was  automatically  marked  on  the  map  to 
make  sure  that  all  anomalies  were  visited. 

5.4.5  Data  handling 

Data  were  stored  as  .tern  files  on  the  field  computer  and  converted  to  .csv  files  before  every 
battery  change.  We  kept  a  copy  of  all  .tern  and  .csv  files  on  the  field  computer,  on  a  portable 
hard-disk  drive  and  on  the  field  laptop  that  was  used  for  reviewing  the  data. 

The  second  operator  documented  the  survey  by  noting  target  names  and  file  numbers  in 
addition  to  any  remarks  made  by  the  principal  operator.  Field  notes  were  digitized  every  day  by 
taking  pictures  of  the  notes  and  filling  out  a  spreadsheet  that  was  used  for  pre-processing. 
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6.0  DATA  ANALYSIS 

6.1  PREPROCESSING 

The  DAQ  recorded  data  streams  from  the  sensor  head,  the  attitude  sensor  and  GPS.  Each 
static  sounding  or  segment  of  a  line  search  is  saved  into  a  .tern  binary  file,  which  is  later 
converted  to  a  .CSV  format  file  without  any  data  alteration.  The  files  were  verified,  renamed  and 
packaged  for  delivery  and  distribution. 

For  the  detection  analysis,  dynamic  data  were  merged  and  the  AHRS  and  GPS  data  were 
combined  to  predict  the  MPV’s  receiver  locations.  For  each  EMI  data  block  in  dynamic  and  cued 
mode,  the  receiver  data  were  divided  by  the  maximum  transmitter  current  amplitude  for  that  data 
block.  This  process  compensates  for  fluctuations  in  transmitter  battery  power  by  normalizing  the 
response  to  a  unit  transmitter  excitation. 

Cued  data  pre-processing  consisted  of  quality  control  to  verify  that  the  sensor  data  was  not 
corrupted  and  that  the  cued  location  was  close  to  the  picked  anomaly  location.  Background 
measurements  were  analyzed  to  define  the  background  response  to  be  subtracted  from  the  cued 
data.  The  resulting  data  were  visually  validated. 

6.2  TARGET  SELECTION  FOR  DETECTION 

Dynamic  survey  data  were  recorded,  filtered  and  interpreted  to  produce  digital  maps  around 
each  house  and  identify  anomalies  that  required  further  investigation.  Anomalies  were  retained 
when  their  signal  amplitude  exceeded  a  given  threshold  which  varied  from  house  to  house. 

The  detection  threshold  was  derived  from  numerical  simulations  of  the  response  to  a  small 
ISO  at  10  inches  depth  at  the  2nd  time  channel  (0.26  ms).  Z-component  data  were  used  for  the 
detection  process  and  for  producing  maps.  Each  receiver  cube  was  processed  as  an  independent 
survey  line  with  an  algorithm  that  picked  targets  along  line  profiles  and  kept  anomalies  for  which 
there  were  at  least  two  consecutive  data  points  exceeding  the  threshold.  The  line  profile 
algorithm  was  preferred  to  the  gridded  image  detection  method  because  the  latter  is  more 
sensitive  to  positional  error  and  data  gaps,  which  can  create  grid  artifacts.  For  each  house,  a 
detection  list  with  geographic  locations  and  anomaly  labels  was  submitted  to  Parsons.  Detection 
maps  for  each  house  are  shown  in  Appendix  B. 

All  seeds  buried  within  the  project  design  specification  of  surface  to  fifteen  centimeter  were 
detected.  Four  additional  seeds  buried  deeper  than  the  design  specification  were  also  detected. 
One  seed,  number  WK-34  buried  at  28  cm  depth  (below  design),  was  not  detected. 

6.3  PARAMETER  ESTIMATION 

Classification  was  based  on  cued  data  measurements.  Data  analysis  was  performed  using 
BTG's  UXOLab,  which  was  developed  with  colleagues  at  the  University  of  British  Columbia 
and  tested  in  numerous  ESTCP  and  SERDP  projects.  When  data  (.CSV  format)  are  imported 
UXOLab  automatically  performs  background  corrections,  using  the  background  that  was 
collected  closest  in  time  for  each  anomaly.  The  data  were  inverted  in  UXOLab  using  a  sequential 
inversion  approach  to  estimate  target  location,  depth  and  primary  polarizabilities.  Instrument 
height  above  the  ground  was  assumed  to  be  7  cm.  Target  location  was  constrained  to  lie  between 
±0.5  m  in  both  X  and  Y  directions  relative  to  the  acquisition  location.  Target  depth  was 
constrained  to  lie  between  -1.2  and  0  m.  The  initial  optimization  for  target  location  identified  up 
to  twelve  starting  models  to  input  into  the  subsequent  estimation  of  polarizabilities.  Three 


BLACK  TUSK  GEOPHYSICS 


23 


Mar.  2017 


ESTCP  MR-201228:  MPV  Demonstration  at  Puako,  HI 


Final  Report 


inversions  were  performed  per  anomaly,  solving  for  (1)  a  single  object  (single  object  inversion: 
SOI);  (2)  two  objects  (201);  and  (3)  three  objects  (301),  and  resulting  in  six  models  each  per 
cued  measurement.  Inversion  results  were  visually  QCed  to  ensure  that  models  derived  from 
inversions  with  poor  data  fits  were  not  included  in  the  classification  process. 

6.4  TRAINING 

Training  data  are  typically  used  to  augment  and  validate  the  reference  ordnance  library  that  is 
used  for  polarizability  matching  during  classification.  For  this  project,  training  data  were  not 
available.  To  compensate  for  a  lack  of  training  data,  a  large  ordnance  library  comprising  628 
items  was  used  for  classification.  Of  these,  617  were  from  recent  ESTCP  test  stand 
measurements  over  an  extensive  range  of  ordnance  types  and  sizes,  including  all  common 
munition  types/size  (20mm,  37mm,  40mm,  57mm,  60mm,  75mm,  81mm,  105mm  155mm, 
grenades,  rockets,  etc.)  and  many  other  less  common  munitions.  The  ordnance  library  also 
contained  eleven  items  based  on  local  test  pit  measurements.  Use  of  this  comprehensive  library 
helped  to  ensure  that  any  one-off  and/or  unexpected  ordnance  would  be  found  through  library 
matching.  Figure  8  shows  the  location  of  the  reference  ordnance  items  in  size  versus  decay 
feature  space,  and  illustrates  the  wide  range  of  size  and  decay  spanned  by  the  reference  items. 


Figure  8:  Example  feature  space  plot  for  house  2  (1765  Puako  Dr.),  showing  distribution  in  decay  versus  size 
feature  space  of  628  reference  features  (stars)  in  relation  to  data  features  (passed  models  are  cyan  dots  while 
failed  models  are  lavender  dots). 
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6.5  CLASSIFICATION 


Classification  is  based  on  an  objective  decision  metric  that  ranks  sources  by  the  misfit 
between  recovered  polarizabilities  with  the  best  matching  reference  polarizabilities  from  the 
reference  library.  The  misfit  (assuming  matching  using  all  three  polarizabilities)  is  calculated  via 
the  following  formula: 


(  N,  V/2 

VH _ , 

\”i  t=l  J 


where  L  are  the  polarizabilities,  w;  is  the  weighting  on  each  polarizability,  A,  specifies  the  range 
of  time  channels,  and  y  is  an  exponential  scaling  (=0.1)  to  ensure  more  equal  contribution  of 
different  time  channels  to  the  misfit  measure.  Misfit  ( <j) )  values  range  between  0  for  a  perfect 
match  to  arbitrarily  large  for  a  bad  match.  The  decision  metric  used  by  the  classifier  is  1 1  (j> .  The 
range  of  time  channels  used  for  matching  was  determined  automatically  based  on  an  objective 
estimation  of  the  reliability  of  the  polarizabilities. 

For  a  given  cued  measurement,  all  models  not  failed  (e.g.,  because  of  a  poor  data  fit)  would 
be  considered  by  the  classifier.  For  a  given  unique  anomaly  number,  all  passed  models  from  all 
cued  measurements  of  the  anomaly  would  be  considered  by  the  classifier,  which  chooses  the 
model  with  the  smallest  polarizability  misfit  for  classification.  The  ranked  dig  list  is  the  list  of 
anomalies  sorted  by  decision  metric  (high  to  low).  Based  on  visual  inspection  of  the  inversion 
results  all  anomalies  in  the  dig  list  were  categorized  as  either  (1)  high  confidence  TOI;  (2)  high 
confidence  non-TOI;  or  (3)  undecided.  All  anomalies  in  the  first  and  third  category  were 
classified  as  digs.  The  remaining  anomalies  were  non-digs.  Separate  dig  lists  were  provided  for 
each  house. 

Classification  was  successful  in  finding  all  seeds.  Results  for  each  house  are  presented  in 
Appendix  B. 
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7.0  COST  ASSESSMENT 

Time  and  resources  were  tracked  for  each  task  to  assess  the  cost  of  deploying  the  technology 
at  future  live  sites.  A  cost  model  is  proposed  in  Table  11,  assuming  an  hourly  rate  of  $100.  The 
field  activities  occurred  over  two  weeks  and  included  one  day  of  instrument  setup,  3  days  of  field 
testing  with  updated  software  and  new  sensor  components,  5.5  days  of  dynamic  survey  and  4.5 
days  of  cued  collection.  The  cost  model  does  not  include  2  additional  members  of  the  Field 
Team  who  were  Parsons  personnel  working  under  a  separate  ESTCP  projcet.  We  assume  $1000 
per  plane  ticket,  a  combined  per  diem  and  lodging  rate  of  $300  per  day  and  $1000  per  week  for 
rental  of  a  pickup  truck.  We  further  assume  that  3  people  were  required  on  the  site  and  do  not 
include  SUXO  personnel  or  escort. 


Table  11:  Cost  model  for  the  MPV  demonstration. 


Cost  Element 

Data  to  be  Tracked 

Unit 

Total 

Hours 

Total  Cost 

Survey  preparation  and  set  up 

$34,800 

Sensor 

maintenance 

Unit:  $  Cost 
•  MPV  maintenance 

$5,000 

Planning 

Personnel:  Geophysicist 
•  Demonstration  plan  and  coordination 

80  h 

$8,000 

Development 

time 

Personnel  required:  Geophysicist 

Time  to  test  target  picking  algorithms  in  the 
presence  of  magnetic  soil 

40  h 

$4,000 

Mobilization 

and 

demobilization 

Cost  to  mobilize  to  site:  1  BTG  Geophysicist 

•  Flight,  hotel,  per  diem  and  time 

•  Shipping 

8  h 

2  h 

16  h 

4  h 

$5,800 

$3,200 

Instrument 

setup 

Typical  field  crew:  1  BTG  Geophysicist 

•  First  day:  assemble,  set  up  and  test  pit 

•  Last  day:  packing 

8  h 

4  h 

8  h 

8  h 

$800 

$800 

Pre-survey 

testing 

Personnel:  pre-deployment  testing  in  Vancouver 
and  initial  on  site  testing 

•  Function  tests  on  all  technology  components 
(hardware  and  software)  and  test  field 
procedures 

8  h 

72  h 

$7,200 

Field  survey:  Daily  tasks  (11  days) 

$12,500 

Rentals, 
materials  and 
miscellaneous 

Survey  equipment  rental  (GPS) 

•  Material  supplies 

•  Travel  to  site,  car  rental  and  gas 

•  Hotel  and  per  diem 

0.5  h 
$300 

2  h 

5.5  h 

$2,500 

$500 

$2,500 

$4,800 

Instrument 

verification 

Field  crew:  BTG  Geophysicist 

•  Typical  day  (GPS  set  up  and  I VS  surveys) 

•  Analyze  IVS  data  (Geophysicist) 

1  h 

1  h 

11  h 

11  h 

$1,100 

$1,100 
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Field  survey:  Detection  (0.72  acre  -  5.5  days) 

$18,100 

Data  collection 

Field  personnel :  BTG  Geophysicist 

for  detection 

•  Collect  &  record  data 

10  h 

55  h 

$5,500 

•  Preparation  and  interruptions 

6  h 

$600 

Detection 

Personnel:  Geophysicist 

processing 

•  Data  extraction  and  QC 

•  Built  detection  map,  establish  threshold  and 

50  h 

$5,000 

pick  anomalies 

50  h 

$5,000 

•  Prepare  data  for  delivery 

20  h 

$2,000 

Field  survey:  Cued  interrogation  (740  anomalies  -  4.5  days) 

$10,400 

Data  collection 

Personnel:  BTG  Geophysicist 

for  cued  survey 

•  Collect  &  record  data 

10  h 

45  h 

$4,500 

•  Preparations  and  interruptions 

1  h/d 

5  h 

$500 

Pre-processing 

Personnel  required:  Geophysicist 

and  QC 

•  Import  and  QC  (per  flag) 

30  h 

$3,000 

•  Prepare  recollects 

8  h 

$800 

•  Prepare  data  for  delivery 

16  h 

$1,600 

Classification  of  cued  interrogation  data  (740  anomalies) 

$9,600 

Data  extraction 

Personnel:  Geophysicist 

Extract  and  analyze  cued  data 

20  h 

$2,000 

Parameter 

Personnel:  Geophysicist 

extraction 

Inversion  setup  &  QC 

20  h 

$2,000 

Classifier 

Personnel:  Geophysicist 

training 

Identify  features  and  potential  TOI 

28  h 

$2,800 

Classification 

Personnel:  Geophysicist 

and  dig  list 

Test  classifier,  prepare  dig  lists  and  assimilate 
groundtruth 

28  h 

$2,800 

COST  SUMMARY 

Dynamic  data  collection  per  property  (incl.  IVS  and  QC) 

$1,200 

Detection  analysis  per  property 

$2,183 

Cued  data  acquisition  per  anomaly  (incl.  IVS  and  QC) 

$8.24 

Cued  data  classification  per  anomaly 

$21.76 
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8.0  MANAGEMENT  AND  STAFFING 

A  flow  chart  showing  the  managerial  hierarchy  and  the  relationship  between  the  principal 
investigator  (PI)  and  other  personnel  is  shown  in  Figure  9. 


Figure  9:  Project  management  structure  for  the  Waikoloa  demonstration. 

The  Puako  study  was  jointly  lead  by  Kevin  Kingdon  (Field/Classification  Geophysicist, 
Black  Tusk  Geophysics)  and  Nicolas  Lhomme  and  Barry  Zelt  (Classification  Geophysicists, 
Black  Tusk  Geophysics).  The  Field  Geophysicist  lead  the  data  collection  on  the  Puako  site, 
trained  the  field  crews  from  Parsons  to  operate  the  technology,  and  took  part  in  the  data 
collection  for  detection  and  cued  interrogation.  After  the  deployment  he  applied  classification  to 
the  MPV3D  data  collected  in  cued  mode  along  with  Barry  Zelt.  The  Classification  Geophysicists 
Lhomme  and  Zelt  advised  on  survey  procedures  and  system  specifications  and  performed  most 
data  analysis  and  data  management  tasks  including:  daily  data  QC  of  the  IVS,  dynamic  and  cued 
data,  processing  of  the  dynamic  data  and  anomaly  picking,  data  packaging  for  distribution, 
processing  of  all  cued  data,  classification  of  the  MPV-3D  data,  retrospective  performance 
analysis  and  reporting.  Laurens  Beran  and  Len  Pasion  of  Black  Tusk  Geophysics  provided 
technical  support  in  establishing  detection  thresholds  and  retrospective  analysis. 
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9.0  IMPLEMENTATION  ISSUES  AND  LESSONS  LEARNED 

There  are  unique  challenges  presented  when  surveying  in  a  semi-urban  environment.  Even  in 
relatively  open  sky  conditions,  RTK  GPS  surveying  can  be  difficult  because  of  infrastructure. 
Future  work  should  consider  robotic  total  station  (RTS)  as  an  option  to  increase  coverage.  RTS 
surveying  also  has  its  own  challenges  as  it  is  often  slow  and  tedious  compared  with  RTK  GPS  as 
multiple  setups  are  often  required  to  maintain  line  of  sight.  One  compromise  might  be  to  use 
RTK  GPS  where  available  and  use  the  alternative  detect,  flag  and  cue  approach  with  the  MPV  as 
described  in  Appendix  B  (MPV  Surveys  at  House  6:  69-1971  Puako  Beach  Drive). 
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APPENDICES 

Appendix  A  Points  of  Contact 

Points  of  contact  (POCs)  involved  in  the  demonstration  and  their  contact  information  are 
presented  in  Table  12. 


Table  12:  Points  of  Contact  for  the  MPV  Demonstration. 


POINT  OF 
CONTACT 

ORGANIZATION 

Phone 

Fax 

E-mail 

Role  in  Project 

Stephen 

Billings 

Black  Tusk  Geophysics 
401-1755,  W  Broadway 
Vancouver,  BC 

V6J  4S5,  Canada 

Tel:  604-428-3382 
stephenbillings  @btgeophy  sics  .com 

Project  PI 

Kevin 

Kingdon 

Black  Tusk  Geophysics 
401-1755,  W  Broadway 
Vancouver,  BC 

V6J  4S5,  Canada 

Tel:  604-428-3380 
kevin.kingdon  @btgeophy  sics  .com 

Project  field 
Geophysicist 

David 

George 

G&G  Sciences,  Inc. 

873  23  Rd 

Grand  Junction,  CO  81505 

Tel:  970-263-9714 
dgeorge  @  ggsciences.com 

Sensor 

manufacturing  and 
support 

Dr.  Herb 
Nelson 

ESTCP  Program  Office 

901  North  Stuart  Street, 
Suite  303 

Arlington,  VA  22203-1821 

Tel:  571-372-6400 
herbert.h.nelsonlO.civ@mail.mil 

ESTCP  MR 
Program  Manager 
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Appendix  B  Detection  and  classification  details  for  the  MPY  study 

The  original  demonstration  plan  (BTG  and  Parsons,  2015)  called  for  surveys  to  be  performed  in 
a  fire  break  area  behind  residential  parcels  as  well  as  within  three  residential  parcels  with  a  main 
objective  being  an  assessment  of  classification  performance  in  a  semi-urban  setting  which  may 
include  increased  cultural  debris,  infrastructure,  and  utilities  all  of  which  may  impact  the  quality 
of  EMI  data.  Rights  of  Entry  (ROE)  issues  could  not  be  resolved  for  the  firebreak  area  prior  to 
the  start  of  MPV  surveys  and  consequently  an  additional  three  residential  properties  with 
existing  ROE  were  selected  for  the  demonstration.  Figure  10  displays  a  map  from  the  Parsons 
demonstration  report  (Parsons  2016)  showing  the  six  properties  surveyed  by  the  MPV.  The  site 
specific  noise  was  found  to  be  variable  from  one  property  to  the  next  and  so  each  residential 
property  was  treated  as  an  independent  survey.  What  follows  is  an  overview  of  the  dynamic  and 
cued  MPV  surveys  for  each  residential  property  with  a  discussion  of  any  site  specific  challenges 
for  each  property  along  with  a  description  of  the  classification  process  and  results. 


Figure  4-1 
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Figure  10:  The  six  residential  properties  included  in  the  Puako  study  (from  Parsons,  2016). 
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B.l  MPV  Surveys  at  House  1:  69-1745  Puako  Beach  Drive 

The  first  residential  property  surveyed  was  located  at  69-1745  Puako  Beach  Drive.  The  dynamic 
surveys  took  place  over  three  days:  December  3  (0.1  hrs),  4  (1.7  hrs)  and  7  (1.5  hrs)  2015.  On  the 
first  day,  only  a  few  transects  were  collected  to  assess  site  specific  noise  levels  after  completing 
IVS  testing.  This  was  followed  by  a  shortened  survey  day  (Dec  4)  after  a  misunderstanding 
between  USACE  and  field  crews  on  the  ability  to  work  on  Fridays.  Only  a  partial  day  (Dec  7) 
was  required  to  complete  dynamic  surveying  for  the  property.  In  general,  it  should  be  possible  to 
survey  in  dynamic  mode  a  property  of  this  size  (assuming  RTK  GPS  signals  can  be  received),  in 
less  than  one  survey  day. 

Photos  of  MPV  dynamic  and  cued  surveying  are  shown  in  Figure  1 1 .  Site  specific  challenges  at 
this  property  included  surveying  as  close  as  physically  possible  to  the  houses  as  shown  in  the  top 
panel  of  Figure  1 1.  It  was  anticipated  that  the  EMI  data  immediately  adjacent  to  houses  would  be 
difficult  to  interpret  and  pick  targets  from  due  to  saturation  of  the  response.  Areas  around 
infrastructure  were  typically  saturated  and  had  to  be  masked  from  target  picking  routines  as 
shown  in  the  bottom  left  panel  of  Figure  12.  The  bottom  left  panel  of  Figure  11  also  illustrated 
some  of  the  vegetation  that  had  to  be  navigated  around  as  well  as  the  overhanging  roofline  that 
blocked  the  GPS  antenna  from  sky  view  when  surveying  near  the  house.  In  the  panel  on  the  right 
of  Figure  11,  the  large  tree  in  the  background  of  the  cued  measurement  created  the  herringbone 
pattern  in  GPSFix  quality  that  can  be  seen  in  the  bottom  right  panel  of  Figure  12.  This  pattern 
was  caused  by  the  straight  line  survey  paths  traversing  in  opposite  directions  underneath  the  tree 
canopy  where  RTK  GPS  positioning  is  temporarily  lost.  The  offset  of  the  GPS  antenna  relative 
to  the  sensor  head  means  that  survey  lines  in  opposite  directions  create  an  alternating  pattern  of 
poor  quality  GPS  data  as  the  GPS  antenna  loses  adequate  satellite  coverage  when  passing  under 
the  tree  canopy  and  takes  time  to  reacquire  satellites  after  emerging  from  underneath  the  tree. 
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Figure  11:  MPY  dynamic  and  cued  surveying  at  house  1:  69-1745  Puako  Beach  Drive. 


Retrospective  analysis  of  the  dynamic  data  in  the  vicinity  of  the  tree  on  the  right  panel  of  Figure 
1 1  led  to  the  development  of  field  techniques  to  minimize  the  degraded  GPSFix  quality  when 
surveying  near  isolated  trees  with  canopy  significant  enough  to  lose  RTK  quality  data.  Taking 
advantage  of  the  offset  of  the  MPV  sensor  head  (which  extends  in  front  of  the  operator)  relative 
to  the  GPS  antenna  (which  resides  behind  the  operator  at  the  top  of  the  MPV  handle)  allows  the 
senor  head  to  be  placed  within  areas  under  a  tree  canopy  while  maintain  RTK  GPS  positioning 
by  keeping  the  GPS  antenna  outside  of  the  tree  canopy.  Approaching  trees  from  a  single 
direction  rather  than  alternating  directions  minimizes  degraded  GPSFix  values  by  extending  the 
sensor  head  as  far  under  the  tree  canopy  as  possible  while  maintaining  GPSFixQ=4.  This 
approach  cannot  eliminate  the  loss  of  RTK  GPS  (particularly  for  large  tree  canopies)  but  it  can 
reduce  the  herringbone  patterns  of  GPSFix  values  illustrated  in  the  bottom  right  panel  of  Figure 
12. 
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Figure  12:  Dynamic  data  overview  for  MPV  surveys  at  69-1745  Puako  Drive.  The  top  left  panel  shows  an 
overhead  view  of  the  property  and  the  survey  environment.  Top  right  panel  overlays  dynamic  MPV  data 
plotted  with  a  color  scale  such  that  values  below  the  site  specific  threshold  for  the  property  are  plotted  in 
grey.  Bottom  left  panel  illustrates  the  gridded  MPV  data  used  for  target  picking  (filtered  channel  2,  0.26  ms) 
as  well  as  the  resulting  230  target  picks.  Bottom  right  panel  shows  a  map  of  GPS  Fix  Q  value. 

Once  MPV  dynamic  data  collection  was  completed  for  this  property,  a  property-specific 
threshold  needed  to  be  determined  for  target  picking.  Expected  TOI  at  the  site  included  a  number 
of  fuzes  which  test  pit  measurements  determined  to  be  relatively  fast  decaying  targets  leading  to 
the  decision  to  choose  an  early  time  channel  for  target  picking.  The  noise  analysis  is  summarized 
in  Figure  13  for  two  different  portions  of  the  property.  A  representative  noise  level  for  the 
property  was  determined  to  be  1.2  mV/A  and  the  threshold  was  set  at  five  times  the  noise  or 
6mV/A  for  target  picking  on  time  channel  2  (0.26ms).  Using  these  parameters  resulted  in  230 
anomalies  being  selected  for  cued  interrogation.  The  appropriateness  of  the  threshold  was 
verified  both  through  modeling  using  the  property-specific  noise  level  with  the  Detection 
Modeler  software  developed  under  SERDP  2226  and  through  empirical  tests  performed  by 
collecting  dynamic  data  over  a  small  ISO  placed  in  the  nearby  test  pit.  The  Detection  Modeler 
was  also  used  to  calculate  a  detection  depth  of  15-20  cm  for  a  small  ISO  based  on  the  property- 
specific  noise  levels.  A  deep  seed  (WK-34,  small  ISO,  horizontal  at  28cm  depth)  was  not 
detected.  Retrospective  analysis  indicates  that  it  was  beyond  the  detection  depth.  The  Detection 
Modeler  calculates  a  threshold  of  0.75mV/A  for  a  horizontal  small  ISO  at  25cm.  With  the 
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property-specific  noise  levels  at  1.2mV/A  and  a  picking  threshold  of  6mV/A,  WK-34  was  a  very 
difficult  target  for  detection.  This  seed  was  beyond  the  detection  objectives  of  the  demonstration 
plan  (a  number  of  seeds  were  intentionally  placed  at  depths  below  the  detection  objective  in 
order  to  provide  challenging  scenarios,  and  because  detection  depths  could  not  be  determined  a 
priori  without  accurate  site  specific  noise  levels).  No  cued  measurements  were  made  over  WK- 
34. 


Noise  on  Channel  2  (0.26  ms): 

•  1.45  mV/A  on  West  side 

*  1.08  mV/A  on  East  side 


East  side  of  house 


Noise  as  a  function  of  time  (Median  Absolute  Deviation) 


West  side  of  house 


Noise  as  a  function  of  time  (Median  Absolute  Deviation) 


Figure  13:  Noise  analysis  for  69-1745  Puako  drive.  Two  different  areas  of  the  property  were  analyzed  to 
arrive  at  a  noise  level  of  1.2  mV/A  for  the  property. 


All  five  seeds  detected  in  dynamic  mode  were  correctly  classified  as  TOI.  A  summary  of  ground 
truth  information  and  inversion  results  are  shown  in 

Figure  14.  The  recovered  polarizabilities  produce  excellent  matches  to  library  items  that  agree 
with  the  ground  truth.  The  bottom  panel  of 

Figure  14  shows  the  results  for  a  deeper  seed,  WK-33  (MPV  target  label  1204,  horizontal  small 
ISO  at  26cm).  It  was  selected  as  a  high  likelihood  TOI  in  the  dig  list  for  this  property  based  on 
its  match  to  a  30mm  TP  library  item.  The  bottom  right  panel  illustrates  that  even  though  the  best 
fitting  library  item  was  the  30mm  TP,  the  recovered  polarizabilities  are  also  an  excellent  match 
to  the  small  ISO  library  item.  Note  that  the  polarizabilities  for  this  deep  seed  are  not  as  well 
constrained,  particularly  at  later  times,  in  comparison  to  the  four  shallower  seeds  in  the  top  panel 
of  Figure  14.  A  comparison  of  the  source  locations  and  depths  predicted  via  inversion  with  the 
ground  truth  values  measured  during  intrusive  operations  is  summarized  in  Figure  15.  Finally, 
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the  ROC  curve  for  this  property  is  shown  in  Figure  16.  Note  that  due  to  budget  and  time 
constraints  on  intrusive  operations,  the  full  ground  truth  past  the  stop  dig  point  was  not  available. 
Ten  items  beyond  the  stop  dig  point  were  intrusively  investigated  and  the  items  removed  were 
confirmed  to  qualitatively  match  the  predicted  results  from  inversion  of  the  MPV3D  data. 


T1016-C9-Mod  1  (Inv  1/3=SOI.  1/1)  JP) 
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V  jmal  iso 
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Figure  14:  Summary  ground  truth  report  illustrating  the  ability  to  correctly  identify  seeds  at  69-1745  Puako 
Drive.  All  detected  seeds  were  correctly  classified  as  TOI.  For  each  seed,  the  recovered  polarizabilities 
obtained  from  inversion  of  the  cued  MPY3D  data  are  illustrated  (red,  black  green  curves)  along  with  the  best 
matching  library  item  (grey  curves).  A  photo  of  the  seed  and  a  summary  of  the  predicted  versus  actual  depths 
of  the  targets  and  offset  are  also  listed  for  each  seed.  Bottom  panel  shows  results  for  a  deep  seed,  WK-33 
(MPY  target  label  1204,  a  horizontal  small  ISO  at  26cm)  that  was  identified  as  high  likelihood  TOI  based  on  a 
match  to  a  30mmTP  library  item  (bottom  left  panel).  It  is  also  a  very  good  match  to  the  small  ISO  library 
item  (bottom  right  panel). 
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Ave  |dx|  =  10.8  cm 
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Measured  depth  (cm) 


Figure  15:  Comparison  of  source  locations  and  depths  predicted  by  inversions  of  MPV3D  data 
from  the  69-1745  Puako  Drive  property  with  the  ground  truth  locations  and  depths  measured 
during  intrusive  operations. 


ground  truth  for  only  10  items  after  stop  dig  point  (red  dot)* 


Figure  16:  ROC  curve  for  the  classification  study  of  69-1745  Puako  Drive  based  on  MPY3D  data.  The 
analyst-defined  stop  dig  point  is  indicated  with  a  red  dot.  The  last  TOI  is  indicated  with  a  blue  dot.  Ground 
truth  was  available  for  only  ten  digs  after  the  stop  dig  point.  The  ROC  curve  assumes  the  remaining  digs  are 
non-TOI. 
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B.2  MPV  Surveys  at  House  2:  69-1765  Puako  Beach  Drive 

The  dynamic  surveys  at  69-1765  Puako  Beach  Drive  took  place  over  two  days:  December  7  (3.3 
hrs)  and  December  8  (0.8  hrs)  2015.  This  property  had  few  trees  in  the  front  yard,  but  there  were 
a  number  of  cultural  features  scattered  throughout  the  property,  some  of  which  are  illustrated  in 
Figure  17.  These  included  numerous  buildings  and  sheds,  multiple  vehicles,  propane  tank,  steel 
clothesline,  a  wood  chipper  connected  to  a  large  tree  by  a  long  metal  chain  lying  on  the  surface, 
overhead  power  lines,  and  a  section  of  solid  lava  rock  with  multiple  metal  tools  on  the  surface. 
There  was  also  an  area  in  the  backyard  with  uniformly  spaced  concrete  footings  interspersed  in  a 
collection  of  native  soil  and  rubble.  A  large  number  of  target  picks  for  this  site  occurred  in  this 
pile  of  native  soil,  rubble  and  concrete  footings  as  shown  in  the  data  image  in  the  bottom  left 
panel  of  Figure  18,  where  the  majority  of  picks  are  clustered  in  that  region  of  the  backyard. 
There  are  also  a  number  of  linear  features  in  the  gridded  dynamic  data  expected  to  correspond  to 
subsurface  utilities  (bottom  left  panel  of  Figure  18)  which  saturated  the  sensor  and  needed  to  be 
masked.  A  large  tree  in  the  backyard  produced  a  herringbone  pattern  in  the  GPS  Fix  Q  values 
(bottom  right  panel  of  Figure  18)  similar  to  what  was  observed  in  the  first  property  surveyed 
(house  1,  69-1745  Puako  Drive).  Implementing  a  similar  dynamic  surveying  method,  where  the 
GPS  obstructing  tree  canopy  is  approached  from  both  sides  with  the  sensor  head  extending  as  far 
as  possible  under  the  tree  canopy  while  keeping  the  GPS  antenna  mounted  on  the  opposite  end  in 
satellite  view,  is  expected  to  reduce  the  degraded  GPS  RTK  values  around  trees  with  moderate 
canopies  in  future  MPV  surveys. 
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Figure  17:  MPV  dynamic  and  cued  surveying  environment  at  house  2:  69-1765  Puako  Beach  Drive. 
Numerous  metallic  cultural  features  were  present  both  around  the  perimeter  of  the  site  and  throughout  the 
survey  area. 
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Figure  18:  Dynamic  data  overview  for  MPV  surveys  at  69-1765  Puako  Drive.  Top  left  panel  shows  an 
overhead  view  of  the  property  and  the  survey  environment.  Top  right  panel  overlays  dynamic  MPV  data 
plotted  with  a  color  scale  such  that  values  below  the  site  specific  threshold  for  the  property  are  plotted  in 
grey.  Bottom  left  panel  illustrates  the  gridded  MPV  data  used  for  target  picking  (filtered  channel  2,  0.26ms) 
as  well  as  the  resulting  194  target  picks.  Bottom  right  panel  shows  a  map  of  GPS  Fix  Q  value. 


The  same  approach  used  at  the  first  property  was  used  at  69-1765  Puako  Drive  to  determine  a 
property  specific  threshold  for  target  picking  with  the  noise  analysis  summarized  in  Figure  19. 
The  noise  at  69-1765  Puako  Drive  (2.5  mV/A)  was  found  to  be  twice  as  large  as  the  noise  levels 
at  the  first  property  (1.2  mV/A).  Using  five  times  the  noise  produced  a  threshold  of  lOmV/A  for 
target  picking  on  the  0.26  ms  time  channel  and  resulted  in  194  anomalies  being  selected  for  cued 
interrogation.  All  linear  saturated  areas,  which  are  expected  to  correspond  to  subsurface  utilities, 
as  well  as  the  asphalt  driveway  were  masked  from  target  picking,  although  five  cued  locations 
were  selected  from  within  these  excluded  areas  to  investigate  the  quality  of  cued  measurements 
in  these  regions. 
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Noise  as  a  function  of  time  (Median  Absolute  Deviation) 


Figure  19:  Noise  analysis  for  69-1765  Puako  Drive.  The  noise  is  twice  as  high  (2.5m V/A)  as  69-1745  Puako 
Drive  (1.2  mV/A). 

The  cued  surveys  took  place  over  three  days:  December  14  (2.4  hrs,  168  total,  105  unique  cued 
measurements),  December  15  (3.8  hrs,  137  total,  89  unique  cued  measurements)  and  December 
17  (0.3  hrs,  2  total,  2  unique  cued  measurements)  2015.  One  of  the  challenges  in  working  in  a 
semi-urban  environment  was  that  cued  measurements  made  near  existing  infrastructure  or 
cultural  targets  often  tended  to  pull  the  sensor  towards  those  targets  based  on  the  infield  QC 
inversion  results.  Operators  would  often  be  conservative  and  take  a  second  cued  measurement  at 
the  predicted  location  suggested  by  the  infield  QC  to  confirm  that  the  inversion  is  pulling  the 
sensor  towards  the  infrastructure  or  cultural  target.  This  approach  was  deemed  prudent  given  that 
this  was  the  first  time  that  MPV  surveys  had  been  attempted  in  residential  areas.  It  was  observed 
that  one  likely  consequence  of  operating  in  a  more  complex  semi-urban  environment  is  a  higher 
recollect  rate. 

Cued  measurements  were  inverted  in  UXOLab  and  classification  was  performed  via  a  library 
matching  method.  Classification  performance  can  be  gauged  by  the  position  of  the  QC  seeds  on 
the  dig  list.  At  69-1765  Puako  Beach  Dr,  all  QC  seeds  were  correctly  flagged  as  high  confidence 
TOI.  Five  of  the  six  QC  seeds  were  reasonably  shallow  relative  to  detections  depths  for  the 
corresponding  seed  items  and  produced  excellent  matches  to  reference  polarizabilities  as  shown 
in  Figure  20.  A  more  challenging  seed  WK-31  (MPV  target  label  2176,  a  horizontal  small  ISO  at 
26cm  depth)  illustrates  how  the  recovered  polarizabilities  are  affected  by  lower  SNR  targets. 
Note  that  the  polarizabilities  at  later  times  for  WK-31  (bottom  right  panel  of  Figure  20)  are  not 
as  well  constrained  as  the  other  shallower,  higher  SNR  QC  seeds.  Even  for  the  low  SNR  WK-31 
seed,  the  recovered  polarizabilities  are  axisymmetric  and  a  good  match  to  the  25mm  TPT  library 
item.  It  is  not  uncommon  for  challenging  deep,  low  SNR  targets  to  match  polarizabilities  for  a 
target  in  a  similar  size  band  rather  than  exactly  matching  the  actual  target  type.  Even  for  this 
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deep  small  ISO  near  the  bounds  of  detectability,  the  inversion  was  able  to  recover  axisymmetric 
polarizabilities  which  matched  a  reference  item  (25mm  TPT)  from  the  same  size  band  as  the 
small  ISO.  Another  key  component  of  the  inversion  results  is  the  accuracy  with  which  they  are 
able  to  recover  target  positions  and  depths.  Accurately  estimating  the  depth  is  important  because, 
for  example,  with  knowledge  that  an  inversion  predicts  a  particular  target  to  be  deep  in  a  low 
SNR  scenario,  the  analyst  can  expect  that  the  secondary  polarizabilities  might  be  poorly 
constrained  and  factor  that  into  determining  an  appropriate  classification  approach  tailored  to  site 
specific  conditions  and  TOI.  A  summary  of  the  predicted  depths  versus  the  ground  truth  values 
for  the  QC  seeds  at  69-1765  Puako  Beach  Drive  is  shown  in  Figure  21.  The  receiver  operating 
characteristic  (ROC)  curve  is  illustrated  in  Figure  22. 
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Figure  20:  Summary  ground  truth  report  illustrating  the  ability  to  correctly  identify  seeds  at  69-1765  Puako 
Drive.  All  seeds  were  detected  and  correctly  classified  as  high  likelihood  TOI.  For  each  seed,  the  recovered 
polarizabilities  obtained  from  inversion  of  the  cued  MPY3D  data  are  illustrated  (red,  black  green  curves) 
along  with  the  best  matching  library  item  (grey  curves).  A  photo  of  the  seed  and  a  summary  of  the  predicted 
versus  actual  depths  of  the  targets  and  offset  are  also  listed  for  each  seed.  Bottom  right  panel  shows  results 
for  a  deep  seed,  WK-31  (MPY  target  label  2176,  a  horizontal  small  ISO  at  26cm)  that  was  identified  as  high 
likelihood  TOI  based  on  a  match  to  a  25mmTP  library  item. 
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Figure  21:  Comparison  of  source  locations  and  depths  predicted  by  inversion  of  MPY3D  data  from  the  69- 
1765  Puako  Drive  property  with  the  ground  truth  locations  and  depths  measured  during  intrusive  operations. 


Number  of  non-TOI  digs 

Figure  22:  ROC  curve  for  the  classification  study  of  69-1765  Puako  Drive  based  on  MPV3D  data.  The 
analyst-defined  stop  dig  point  is  indicated  with  a  red  dot.  The  last  TOI  is  indicated  with  a  blue  dot. 
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B.3  MPV  Surveys  at  House  3:  69-1853  Puako  Beach  Drive 

The  dynamic  surveys  at  69-1853  Puako  Beach  Drive  took  place  over  a  single  day,  December  8, 
2015.  Dynamic  surveying  of  the  property  took  only  1.6  hrs  as  a  large  portion  of  the  backyard  for 
this  property  was  comprised  of  a  large  pond.  This  property  had  a  mostly  open  front  yard  with  a 
few  trees,  and  cultural  features  scattered  throughout  the  property,  some  of  which  are  illustrated  in 
Figure  23.  These  included  a  large  satellite  dish,  outdoor  shower  area,  covered  patios  with 
reinforced  concrete,  overhanging  rooflines  that  obscured  the  GPS  signal,  as  well  as  the  large 
pond  feature.  An  overhanging  roofline  in  the  backyard  produced  a  herringbone  pattern  in  the 
GPS  Fix  Q  values  (bottom  right  panel  of  Figure  24)  similar  to  what  was  observed  in  the  first 
property  surveyed,  69-1745  Puako  Drive.  Implementing  a  similar  dynamic  surveying  method, 
where  the  GPS  obstructing  roofline  of  the  house  is  approached  from  a  single  direction  with  the 
sensor  head  extending  as  far  as  possible  under  the  roofline  while  keeping  the  GPS  antenna 
mounted  on  the  opposite  end  of  the  MPV  in  satellite  view,  is  expected  to  reduce  the  degraded 
GPS  RTK  values  around  this  type  of  obstruction  in  future  MPV  surveys. 


Figure  23:  MPV  dynamic  and  cued  surveying  environment  at  house  3:  69-1853  Puako  Beach  Drive.  A  small 
survey  area  with  a  large  pond  comprising  most  of  the  back  yard.  Trees  and  some  cultural  targets  are  present 
in  the  survey  area. 
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Figure  24:  Dynamic  data  overview  for  MPY  surveys  at  69-1853  Puako  Drive.  Top  left  panel  shows  an 
overhead  view  of  the  property  and  the  survey  environment.  Top  right  panel  overlays  dynamic  MPY  data 
plotted  with  a  color  scale  such  that  values  below  the  site-specific  threshold  for  the  property  are  plotted  in 
grey.  The  bottom  left  panel  illustrates  the  gridded  MPV  data  used  for  target  picking  (filtered  channel  2, 
0.26ms)  as  well  as  the  resulting  51  target  picks.  Bottom  right  panel  shows  a  map  of  GPS  Fix  Q  value. 

The  property  specific  noise  level  estimate  for  69-1853  Puako  Drive  was  2.2  mV/A  for  the  0.26 
ms  detection  channel  as  shown  in  Figure  25.  A  detection  threshold  of  lOmV/A  was  used  for 
target  picking  on  the  0.26  ms  time  channel  and  resulted  in  51  anomalies  being  selected  for  cued 
interrogation.  All  saturated  areas  were  masked  and  excluded  from  target  picking  as  shown  in  the 
bottom  left  panel  of  Figure  24. 
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Noise  as  a  function  of  time  (Median  Absolute  Deviation) 


Figure  25:  Noise  analysis  for  69-1853  Puako  drive.  The  noise  is  2.2mV/A  at  the  detection  channel  (Channel  2, 

0.26ms). 


The  cued  surveys  at  69-1853  Puako  Beach  Drive  took  place  on  December  15,  2015  when  77 
cued  measurements  were  made  over  51  unique  anomalies  in  1.5  hours  of  cued  survey  time. 
There  are  more  cued  measurements  (77)  than  target  picks  (51)  because  the  operator  would  often 
collect  a  second  measurement  based  on  the  results  of  the  infield  single  object  inversion  of  the 
cued  data. 

Cued  measurements  were  inverted  in  UXOLab  and  classification  was  performed  via  a  library 
matching  method.  Classification  performance  can  be  gauged  by  the  position  of  the  QC  seeds 
within  the  dig  list.  At  this  property,  all  detected  QC  seeds  were  correctly  flagged  as  high 
confidence  TOI.  One  QC  seed  at  the  property  (WK-09)  was  not  covered  by  dynamic  surveys 
because  it  was  in  a  garden  area  on  the  far  side  of  a  driveway  that  was  blocked  by  parked 
vehicles.  Three  of  the  five  QC  seeds  were  reasonably  shallow  relative  to  detections  depths  for  the 
corresponding  seed  items  and  produced  excellent  matches  to  reference  polarizabilities  as  shown 
in  the  left  panel  of  Figure  26.  All  three  of  these  QC  seeds  produced  recovered  polarizabilities 
with  a  best  match  to  the  correct  library  items  (i.e.  medium  ISO,  MKII  hand  grenade  and  small 
ISO)  from  the  extensive  library.  Polarizabilities  for  two  more  challenging  deep  seeds  are  shown 
in  the  right  panel  of  Figure  26:  WK-35  (  MPV  target  label  3019,  a  horizontal  small  ISO  at 
28cm)  on  top  right  and  WK-36  (MPV  target  label  3054,  a  horizontal  small  ISO  at  28cm)  on 
bottom  right.  Both  were  flagged  as  high  likelihood  TOI  by  the  analyst  in  spite  of  having 
degraded  matches  to  the  library  item  because  the  depth  of  these  targets  from  the  inversions 
suggested  that  polarizabilities,  particularly  the  secondary  polarizabilities  were  likely  to  be  poorly 
constrained  due  to  the  depth  of  the  target  being  near  the  detectability  and  classification  limits  of 
the  technology.  For  WK-36,  the  primary  polarizability  is  a  good  match  to  the  small  ISO 
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reference.  For  WK-35,  the  target  was  selected  as  high  likelihood  TOI  because  of  a  polarizability 
match  to  a  57mm  polarizability  from  the  reference  library.  The  inability  to  recover 
polarizabilities  that  match  the  ground  truth  of  a  small  ISO  for  WK-35  is  a  consequence  of 
operating  in  the  low  SNR  environment  of  targets  below  the  modelled  detection  and  classification 
depths.  In  this  case  the  inversion  models  a  slightly  larger  target  (57mm)  at  a  deeper  depth 
(33.7cm).  A  summary  of  the  predicted  depths  versus  the  ground  truth  values  for  the  QC  seeds  is 
shown  in  Figure  27.  The  receiver  operating  characteristic  (ROC)  curve  is  illustrated  in  Figure  28. 
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Figure  26:  Summary  ground  truth  report  illustrating  ability  to  correctly  identify  seeds  at  69-1853  Puako 
Drive.  All  seeds  detected  were  correctly  classified  as  high  likelihood  TOI.  For  each  seed,  the  recovered 
polarizabilities  obtained  from  inversion  of  the  cued  MPY3D  data  are  illustrated  (red,  black  green  curves) 
along  with  the  best  matching  library  item  (grey  curves).  A  photo  of  the  seed  and  a  summary  of  the  predicted 
versus  actual  depths  of  the  targets  and  offset  are  also  listed  for  each  seed. 
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Figure  27:  Comparison  of  source  locations  and  depths  predicted  by  inversion  of  MPY3D  data  from  the  69- 
1853  Puako  Drive  property  with  the  ground  truth  locations  and  depths  measured  during  intrusive  operations. 


Number  of  non-TOI  digs 

Figure  28:  ROC  curve  for  the  classification  study  of  69-1853  Puako  Drive  based  on  MPV3D  data. 
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B.4  MPV  Surveys  at  House  4:  69-1877  Puako  Beach  Drive 

The  dynamic  surveys  at  69-1877  Puako  Beach  Drive  took  place  over  two  days,  December  8-9, 
2015.  Dynamic  surveying  of  the  property  took  only  1.5  hrs  as  a  large  portion  of  the  backyard  for 
this  property  was  under  a  substantial  tree  canopy  (see  upper  left  panel  of  Figure  30)  and  could 
not  be  surveyed  with  RTK  GPS  positioning.  Two  alternative  approaches  to  dynamic  MPV 
surveying  in  areas  without  RTK  GPS  positioning  were  attempted  in  the  backyard.  A  detailed 
explanation  of  the  two  alternative  MPV  surveying  approaches  is  described  in  section  B.6  for  the 
69-1971  Puako  Drive  property  which  also  contained  overhead  tree  canopy  over  virtually  the 
entire  yard  making  RTK  GPS  surveying  impossible. 


The  69-1877  Puako  Beach  Drive  property  had  a  front  yard  with  a  few  trees,  a  tall  hedge  along 
one  boundary  of  the  property,  and  cultural  features  scattered  throughout  the  property,  some  of 
which  are  illustrated  in  Figure  29.  These  included  multiple  vehicles,  an  office  trailer  and 
overhead  power  lines.  A  large  hedge  in  the  front  yard  produced  degraded  GPS  Fix  Q  values 
(bottom  right  panel  of  Figure  30)  on  alternating  survey  lines.  Implementing  a  similar  dynamic 
surveying  method,  where  the  GPS  obstructing  hedge  is  approached  from  a  single  direction  with 
the  sensor  head  extending  as  far  as  possible  under  the  hedge  while  keeping  the  GPS  antenna 
mounted  on  the  opposite  end  of  the  MPV  in  satellite  view,  is  expected  to  improve  RTK  GPS 
coverage  in  future  MPV  surveys. 


Figure  29:  MPV  dynamic  and  cued  surveying  environment  at  house  4:  69-1877  Puako  Beach  Drive. 
Significant  tree  canopy  in  the  backyard  meant  RTK  GPS  surveying  was  not  possible.  Trees,  vehicles,  an  office 
trailer  and  overhead  power  lines  were  some  of  the  cultural  targets  present  in  the  survey  area. 
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Figure  30:  Dynamic  data  overview  for  MPY  surveys  at  69-1877  Puako  Drive.  Top  left  panel  shows  an 
overhead  view  of  the  property  and  the  survey  environment.  Top  right  panel  overlays  dynamic  MPY  data 
plotted  with  a  color  scale  such  that  values  below  the  site-specific  threshold  for  the  property  are  plotted  in 
grey.  Bottom  left  panel  illustrates  the  gridded  MPV  data  used  for  target  picking  (filtered  channel  2,  0.26ms) 
as  well  as  the  resulting  60  target  picks.  Bottom  right  panel  shows  a  map  of  GPS  Fix  Q  value.  Note  the 
backyard  GPS  Fix  Q  values  illustrate  that  RTK  GPS  surveying  was  not  possible  due  to  significant  tree 
canopy.  Alternative  MPV  detect  and  flag  methods  were  attempted  in  this  area  . 


The  property  specific  noise  level  estimate  for  69-1877  Puako  Drive  was  of  1.72  mV/A  for  the 
0.26  ms  detection  channel  as  shown  in  Figure  31.  A  detection  threshold  of  8.5  mV/A  was  used 
for  target  picking  on  the  0.26  ms  time  channel  and  resulted  in  60  anomalies  being  selected  for 
cued  interrogation.  All  saturated  areas  were  masked  and  excluded  from  target  picking  as  shown 
in  the  bottom  left  panel  of  Figure  30.  Target  picks  were  not  submitted  for  the  entire  backyard 
area  as  there  was  not  sufficient  positional  accuracies  without  RTK  GPS  signal  to  create  a  map 
and  pick  targets. 


BLACK  TUSK  GEOPHYSICS 


51 


Mar.  2017 


ESTCP  MR-201228:  MPV  Demonstration  at  Puako,  HI 


Final  Report 


Noise  as  a  function  of  time  (Median  Absolute  Deviation) 


Figure  31:  Noise  analysis  for  69-1877  Puako  drive.  The  noise  is  1.72mV/A  at  the  detection  channel  (Channel 

2,  0.26ms). 


The  cued  surveys  at  the  front  yard  of  69-1877  Puako  Beach  Drive  took  place  on  December  15, 
2015  when  68  cued  measurements  were  made  over  47  unique  anomalies  in  1.1  hours  of  cued 
survey  time.  On  the  final  day  of  surveying  (December  17),  cued  measurements  were  taken  in  the 
backyard  using  a  detect,  flag  and  cue  approach  whereby  the  MPV  was  initially  used  in  detection 
mode  to  walk  rope  lanes  and,  using  the  dancing  arrows  display  of  EM3D,  identify  targets  in  real 
time  that  were  then  flagged  for  immediate  cueing  with  the  MPV3D  after  completion  of  the 
dancing  arrow  detection  survey.  This  alternative  survey  method  did  show  promise  for  scenarios 
where  accurate  positioning  information  is  not  available  as  a  number  of  cued  measurements  in  the 
backyard  produced  results  in  agreement  with  emplaced  seeds.  The  lack  of  RTK  GPS  signal 
throughout  the  backyard  and  the  direction  from  USACE  to  not  leave  flags  to  guide  later  intrusive 
investigations  led  to  the  backyard  area  being  excluded  from  the  official  investigations. 

Cued  measurements  were  inverted  in  UXOLab  and  classification  was  performed  via  a  library 
matching  method.  Classification  performance  can  be  the  position  of  the  QC  seeds  within  the  dig 
list.  At  69-1877  Puako  Beach  Drive,  all  detected  QC  seeds  were  correctly  flagged  as  high 
confidence  TOI.  All  three  of  these  QC  seeds  produced  recovered  polarizabilities  with  a  best 
match  to  the  correct  library  items  from  the  extensive  library  (Figure  31).  A  summary  of  the 
predicted  depths  versus  the  ground  truth  values  for  the  QC  seeds  is  shown  in  Figure  33.  The 
receiver  operating  characteristic  (ROC)  curve  is  illustrated  in  Figure  34. 
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T4003-C38-Mod  2  (Inv  2/3=201:  1/2)  [P] 


TYPE:  TOI 
ID:  Small  ISO 

DEPTH:  7.0  /  PREDICTED:  3.6 
OFFSET  15.6  I  LENGTH:  10.0 


T4028-C41-Mod  2  (Inv  2/3=201:  1/2)  [P] 


TYPE  TOI 

ID:  20- mm  projectile 

DEPTH  3.0  /  PREDICTED:  2.7 

OFFSET  11.5  /  LENGTH:  8.0 


T4506-C78-Mod  3  (Inv  2/3=201:  2/2)  [P] 


TYPE:  TOI 
ID:  Medium  ISO 

DEPTH:  20.0  /  PREDICTED  21  2 
OFFSET:  19.9  /  LENGTH:  20.5 


Figure  32:  Summary  ground  truth  report  illustrating  ability  to  correctly  identify  seeds  at  69-1877  Puako 
Drive.  All  seeds  detected  were  correctly  classified  as  high  likelihood  TOI.  For  each  seed,  the  recovered 
polarizabilities  obtained  from  inversion  of  the  cued  MPY3D  data  are  illustrated  (red,  black  green  curves) 
along  with  the  best  matching  library  item  (grey  curves)  are  shown.  A  photo  of  the  seed  and  a  summary  of  the 
predicted  versus  actual  depths  of  the  targets  and  offset  are  also  listed  for  each  seed. 
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Figure  33:  Comparison  of  source  locations  and  depths  predicted  by  inversions  of  MPY3D  data  from  the  69- 
1877  Puako  Drive  property  with  the  ground  truth  locations  and  depths  measured  during  intrusive  operations. 


Number  of  non-TOI  digs 

Figure  34:  ROC  curve  for  the  classification  study  of  69-1877  Puako  Drive  based  on  MPV3D  data. 
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B.5  MPV  Surveys  at  House  5:  69-1951  Puako  Beach  Drive 

The  dynamic  surveys  at  69-1951  Puako  Beach  Drive  took  place  on  December  9,  2015  over  a 
period  of  3  hours.  This  property  had  a  front  yard  with  a  few  trees  and  cultural  features  scattered 
throughout  the  property  including  those  illustrated  in  Figure  29.  These  included  multiple 
vehicles,  an  office  trailer  and  overhead  power  lines.  A  large  tree  in  the  back  yard  produced 
degraded  GPS  Fix  Q  values  (bottom  right  panel  of  Figure  24  Figure  36).  There  were  also  linear 
saturated  areas  that  extended  from  the  house  into  both  the  front  yard  and  the  back  yard.  This  was 
assumed  to  be  a  metallic  subsurface  utility.  Areas  of  saturated  sensor  response  were  masked  for 
target  picking  purposes  as  shown  in  the  bottom  left  panel  of  Figure  36. 


Figure  35:  MPV  dynamic  and  cued  surveying  environment  at  house  5:  69-1951  Puako  Beach  Drive.  Trees, 
metal  fence  parts  and  overhead  power  lines  were  some  of  the  cultural  features  present  in  the  survey  area. 
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Figure  36:  Dynamic  data  overview  for  MPY  surveys  at  69-1951  Puako  Drive.  Top  left  panel  shows  an 
overhead  view  of  the  property  and  the  survey  environment.  Top  right  panel  overlays  dynamic  MPY  data 
plotted  with  a  color  scale  such  that  values  below  the  site  specific  threshold  for  the  property  are  plotted  in 
grey.  Bottom  left  panel  illustrates  the  gridded  MPV  data  used  for  target  picking  (filtered  channel  2,  0.26ms) 
as  well  as  the  resulting  96  target  picks.  Bottom  right  panel  shows  a  map  of  GPS  Fix  Q  value. 


The  property  specific  noise  level  estimate  for  69-1951  Puako  Drive  was  of  2.1  mV/A  for  the  0.26 
ms  detection  channel  as  shown  in  Figure  31.  A  detection  threshold  of  10  mV/A  was  used  for 
target  picking  on  the  0.26  ms  time  channel  and  resulted  in  96  anomalies  being  selected  for  cued 
interrogation.  All  saturated  areas  were  masked  and  excluded  from  target  picking  as  shown  in  the 
bottom  left  panel  of  Figure  36. 
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Noise  as  a  function  of  time  (Median  Absolute  Deviation) 


Figure  37:  Noise  analysis  for  69-1951  Puako  drive.  The  noise  is  2.1mV/A  at  the  detection  channel  (Channel  2, 

0.26ms). 


The  cued  surveys  at  69-1951  Puako  Beach  Drive  took  place  over  two  days:  December  16-17, 
2015.  On  December  16,  143  total  and  92  unique  cued  measurements  were  made  over  2.4  hours 
of  cued  MPV3D  surveying.  Field  crews  returned  on  December  17  to  collect  an  additional  13 
total  and  9  unique  cued  measurements  over  0.4  hours. 

Cued  measurements  were  inverted  in  UXOLab  and  classification  was  performed  via  a  library 
matching  method.  Classification  performance  can  be  gauged  by  the  position  of  QC  seeds  within 
the  dig  list.  At  69-1951  Puako  Beach  Dr,  all  QC  seeds  were  correctly  flagged  as  high  confidence 
TOI.  All  six  QC  seeds  were  reasonably  shallow  relative  to  detection  depths  and  produced 
excellent  matches  to  reference  polarizabilities  as  shown  in  Figure  38.  A  summary  of  the 
predicted  depths  versus  the  ground  truth  values  for  the  QC  seeds  is  shown  in  Figure  39.  The 
high-quality  polarizabilities  obtained  for  these  relatively  shallow  targets  led  to  a  very  efficient 
classification  as  indicated  by  the  receiver  operating  characteristic  (ROC)  curve  shown  in  Figure 
40  where  the  first  six  items  on  the  dig  list  correspond  to  the  six  QC  seeds. 
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Figure  38:  Summary  ground  truth  report  illustrating  ability  to  correctly  identify  seeds  at  69-1951  Puako 
Drive.  All  seeds  were  detected  and  correctly  classified  as  high  likelihood  TOI.  For  each  seed,  the  recovered 
polarizabilities  obtained  from  inversion  of  the  cued  MPY3D  data  are  illustrated  (red,  black  green  curves) 
along  with  the  best  matching  library  item  (grey  curves).  A  photo  of  the  seed  and  a  summary  of  the  predicted 
versus  actual  depths  of  the  targets  and  offset  are  also  listed  for  each  seed. 


Ave  |dx|  =  8.3  cm 
Ave  |dy|  =  1.1  cm 
Ave  dr  =  8.4  cm  (N=6) 


Median  error  =  1.9  cm 


Measured  depth  (cm) 


Easting  error  (cm) 

Figure  39:  Comparison  of  source  locations  and  depths  predicted  by  inversion  of  MPY3D  data  from  the  69- 
1951  Puako  Drive  property  with  the  ground  truth  locations  and  depths  measured  during  intrusive  operations. 
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Figure  40:  ROC  curve  for  the  classification  study  of  69-1951  Puako  Drive  based  on  MPV3D  data. 


B.6  MPV  Surveys  at  House  6:  69-1971  Puako  Beach  Drive 

The  property  at  69-1971  Puako  Beach  Drive  presented  the  most  difficult  environment  for 
positioning  due  to  the  lack  of  RTK  GPS  signal  over  virtually  the  entire  property.  The  property 
would  have  been  an  ideal  candidate  for  robotic  total  station  (RTS)  surveying  because,  although 
there  was  a  significant  overhead  canopy,  the  large  trees  were  almost  exclusively  around  the 
perimeter  of  the  property  allowing  for  a  clear  line  of  sight  for  the  majority  of  the  property  (see 
Figure  41).  Without  access  to  an  RTS  system  during  the  demonstration,  alternative  survey 
techniques  were  experimented  with  during  the  demo  to  assess  the  potential  for  MPV  surveying  in 
environments  of  degraded  positional  accuracy.  The  futility  of  accurately  mapping  areas  without 
RTK  GPS  accuracy  is  demonstrated  in  Figure  42  where  a  small  area  was  surveyed  to  investigate 
alternative  positioning  approaches.  One  meter  wide  line  rope  lanes  were  laid  out  for  the  MPV  to 
survey  at  tightly  controlled  0.5  m  line  spacing.  Even  though  the  sensor  head  was  constrained  by 
the  rope  lanes  to  travel  straight  lines  separated  by  0.5m,  the  positions  as  reported  by  the  degraded 
GPS  data  in  Figure  42  do  not  come  close  to  resembling  the  actual  path  of  the  MPV  sensor  head. 


BLACK  TUSK  GEOPHYSICS 


59 


Mar.  2017 


ESTCP  MR-201228:  MPV  Demonstration  at  Puako,  HI 


Final  Report 


Figure  41:  MPY  dynamic  and  cued  surveying  environment  at  69-1971  Puako  Beach  Drive.  While  the  yard 
itself  was  relatively  unobstructed,  large  trees  around  the  perimeter  created  a  canopy  that  covered  virtually 
the  entire  property  making  RTK  GPS  surveying  impossible. 
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Figure  42:  Gridded  image  of  detection  channel  for  69-1971  Puako  Drive  when  surveying  underneath 
significant  tree  canopy. 

An  alternative  positioning  method  was  attempted  on  the  small  test  area  of  degraded  GPS  quality 
data.  In  addition  to  the  rigid  0.5  m  line  spacing  resulting  from  the  rope  lanes,  line  lengths  were 
measured  and  recorded  in  the  field  notes.  The  MPV  operator  attempted  to  walk  the  fixed  length 
survey  lines  at  a  constant  speed.  Normally  the  operator  would  have  the  survey  speed  displayed 
on  the  display  to  provide  feedback  that  could  assist  in  maintaining  a  constant  speed  but  without 
valid  positioning  data,  the  speed  values  reported  are  inaccurate.  The  intention  was  that  tight 
control  on  line  spacing,  survey  speed  and  line  lengths  could  be  exploited  to  produce  a  reasonably 
accurate  map  from  which  to  pick  targets  and  return  to  those  locations  for  cued  MPV3D 
measurements.  The  dancing  arrows  could  be  used  to  ensure  the  MPV  was  properly  positioned  on 
top  of  a  legitimate  metallic  source  to  overcome  inherent  positional  inaccuracies  in  the  detection 
map.  A  parametric  correction  approach  was  devised  to  position  the  data  using  the  measured  line 
lengths,  the  fixed  line  spacing  and  directions  to  constrain  the  data  to  follow  the  known  and 
measured  survey  area.  The  resulting  corrected  positions  and  the  resulting  gridded  map  of  the 
0.26  ms  detection  channel  data  are  depicted  in  Figure  43. 
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Figure  43:  Degraded  GPS  quality  survey  line  positions  after  applying  a  parametric  correction  approach  (on 
the  left)  and  the  corresponding  sensor  data  (on  right). 


The  property  specific  noise  at  69-1971  Puako  Drive  was  the  highest  of  all  the  properties 
surveyed:  3mV/A.  Using  a  threshold  of  12mV/A  resulted  in  87  target  picks  for  cued 
measurements.  This  property  contained  areas  of  relatively  high  target  densities  making  it 
difficult  to  confidently  return  to  targets  identified  in  the  crude  map  given  the  positional 
inaccuracies  inherent  in  the  cued  map.  This  survey  method  was  ultimately  abandoned  when  the 
homeowner  required  that  the  rope  lanes  be  removed  to  allow  for  mowing  of  the  lawn  by 
contractors. 

A  simpler  alternative  positioning  method  was  attempted  whereby  the  same  0.5-m-wide  rope 
survey  lanes  were  laid  out.  The  MPV  operator  then  walked  those  lines  in  dynamic  mode 
observing  the  dancing  arrows  to  identify  targets  in  real  time  on  the  MPV  display.  When  a  target 
was  identified,  a  flag  was  placed  in  the  ground  and  the  operator  continued  walking  the 
predefined  survey  lanes,  identifying  targets  based  on  the  EM3D  dancing  arrows  display.  The 
entire  yard  was  surveyed  using  this  approach  and  70  targets  were  identified.  After  completing  the 
dynamic  mode  search  of  the  rope  target  lanes,  the  horizontal  coils  were  added  and  MPV3D 
measurements  were  made  over  each  of  the  70  targets  identified.  The  lack  of  accurate  positioning 
data  for  these  cued  measurements  was  a  major  obstacle  to  intrusive  operations,  especially  since 
field  teams  were  instructed  flags  could  not  be  left  in  the  ground.  Distances  and  headings  were 
measured  from  fixed  points  (i.e.  SW  corner  post  of  shed,  SE  patio  foundation  block)  and 
recorded  in  field  notes  to  potentially  guide  future  intrusive  operations.  Ultimately  due  to  limited 
budget  and  time  constraints,  intrusive  operations  were  focused  on  the  properties  with  high 
quality  positioning  information  and  no  intrusive  investigations  were  performed  at  69-1971  Puako 
Drive. 
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Figure  44:  Example  of  field  notes  taken  to  position  the  MPV3D  cued  measurements.  A  distance  and  heading 
were  recorded  from  a  range  of  fixed  points  on  the  property. 


In  order  to  evaluate  performance  of  the  detect,  flag  and  cue  approach  in  areas  of  degraded 
positional  data,  the  eight  QC  seeds  placed  on  the  property  were  investigated.  In  order  to  locate 
cued  MPY3D  locations  on  the  map,  approximate  GPS  positions  were  obtained  from  Google 
Earth  for  the  fixed  reference  points  on  the  properties  as  shown  in  Figure  45.  It  is  acknowledged 
that  this  process  introduces  inherent  inaccuracies  as  the  ability  to  accurately  select  the  reference 
point  from  an  aerial  view  of  the  property  has  limitations  (as  does  the  distance/heading 
measurements  made  from  the  fixed  reference  points  on  the  property).  The  goal  here  was  not  to 
obtain  sufficient  accuracies  to  confidently  co-locate  the  MPY3D  cued  measurements  with  the 
reported  seed  locations  but  rather  to  indicate  if  cued  locations  were  made  in  the  vicinity  of  the 
reported  seed  locations.  If  cued  measurements  were  found  to  be  located  near  the  reported  seed 
locations,  the  recovered  polarizabilities  and  target  depths  could  then  be  investigated  to  add 
confidence  that  the  MPV  correctly  detected  and  the  MPV3D  correctly  classified  the  QC  seeds. 
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J  Reference  points 

1  =  Southwest  corner  post  of  shed  (studio) 

2  =  Northwest  post  of  patio  staircase 

3  =  Northeast  patio  foundation  block 

4  =  Southeast  patio  foundation  block 
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Figure  45:  GPS  co-ordinates  for  reference  points  used  by  field  crews  to  position  cued  MPV3D  measurements 
were  identified  in  Google  Earth  in  order  to  investigate  performance  against  QC  seeds  at  69-1971  Puako 
Drive. 


Figure  46  plots  a  comparison  of  the  estimated  locations  of  items  flagged  as  potential  TOI  along 
with  the  reported  locations  of  seeds.  For  reference  and  scale,  the  gridded  data  corresponding  to 
the  corrected  positions  (from  Figure  43)  is  plotted.  The  challenge  of  creating  and  interpreting  a 
detection  map  without  accurate  positional  information  is  reinforced  by  noting  that  none  of  the 
seed  locations  correspond  to  an  area  of  elevated  amplitude  on  the  map.  The  relative  positions  of 
the  flagged  TOI  and  seed  locations  are  well  correlated  and  appear  to  be  offset  by  constant  shifts 
which  are  likely  caused  by  inaccuracies  described  earlier  related  to  estimating  positions  of  cued 
measurements.  The  recovered  polarizabilities  for  targets  flagged  as  possible  TOI  are  shown  in 
Figure  47.  All  QC  seeds  at  the  property  were  relatively  shallow  small  ISOs.  Seven  of  the  eight 
items  flagged  as  high  likelihood  TOI  that  were  in  the  vicinity  of  seed  locations  have  recovered 
polarizabilities  that  are  excellent  matches  to  small  ISO  polarizabilities.  The  eighth  item  was  not 
flagged  as  a  high  likelihood  TOI  but  was  placed  in  the  “gray  area”  targets  that  were  also 
categorized  as  dig.  This  particular  item,  target  6080  does  not  have  an  excellent  match  to 
reference  polarizabilities  (and  hence  was  not  classified  as  TOI)  but  does  have  a  primary 
polarizability  decay  that  is  a  good  match  to  a  small  ISO  at  late  times  which  is  why  it  was  placed 
in  the  dig  category.  It  is  suspected  that  target  6080  corresponds  to  seed  WK-18.  A  comparison 
of  ground  truth  and  recovered  depths  from  cued  MPV3D  measurements  in  Table  13  provides 
further  support  that  the  seeds  have  been  detected  and  classified  correctly  as  the  average  depth 
error  is  only  2.7cm. 
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Figure  46:  Comparison  of  the  estimated  locations  for  items  flagged  as  high  likelihood  TOI  during  data  QC 
(yellow  dots)  and  seed  locations  (triangles).  Target  6080  was  included  in  the  “grey  area  anomalies”  which 
were  selected  to  be  dug  but  were  not  flagged  as  high  confidence  TOI.  It  is  suspected  that  anomaly  6080 
corresponds  to  seed  WK-18. 


Figure  47:  Polarizabilities  for  targets  flagged  as  high  likelihood  small  ISO  during  QC  (1-7).  Item  (8)  shows 
polarizabilities  for  an  item  not  flagged  as  high  likelihood  TOI  but  still  classified  as  a  dig.  It  is  suspected  that 
this  item  corresponds  to  seed  WK-18  (small  ISO).  The  blue  labels  represent  best  guesses  at  corresponding 
seed  IDs. 
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Target 

Seed  ID 

Actual  Z 
(cm) 

Model  Z  (cm) 

Z  error 
(cm) 

Actual 

inclination 

Model 

inclination 

6001 

22 

7 

4.0 

-3.0 

0 

1 

6002 

19 

8 

6.2 

-1.8 

45 

54 

6004 

16 

8 

5.6 

-2.4 

45 

61 

6007 

14 

6 

4.3 

-1.7 

45 

56 

6011 

10 

5 

2.6 

-2.4 

90 

98 

6013 

23 

8 

3.7 

-4.3 

45 

52 

6070 

11 

6 

2.6 

-3.4 

90 

90 

6080 

18 

5 

2.1 

-2.9 

90 

5 

Table  13:  Comparison  of  recovered  depth  for  cued  MPV3D  measurements  flagged  as  high  likelihood  small 
ISO  and  the  ground  truth  depths 


There  are  three  targets  in  Figure  46  (6022,  6023,  6032)  that  were  flagged  as  high  likelihood  TOI 
yet  did  not  correspond  to  any  seed  locations.  Polarizabilities  for  these  items  are  shown  in  Figure 
48.  All  three  of  these  items  have  very  suspicious  polarizabilities,  but  their  locations  (and 
size/shape)  do  not  correspond  to  any  of  the  QC  seeds. 
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Figure  48:  Polarizabilities  for  three  targets  flagged  as  high  likelihood  TOI  during  QC. 


The  MPV  detect,  flag  and  cue  approach  seems  to  show  promise  for  surveying  in  areas  where 
accurate  positional  data  cannot  be  obtained.  The  approach  appears  to  have  identified  all  QC 
seeds  but  it’s  worth  noting  that  all  seeds  were  relatively  shallow  and  therefore  straightforward  to 
detect.  It’s  unclear  how  well  this  approach  would  work  with  deeper  more  challenging  seeds.  One 
drawback  of  this  approach  is  that  the  process  is  subjective  and  would  likely  vary  based  on 
operator  experience.  The  process  could  be  made  less  subjective  by  specifying  the  required 
sensitivity  of  the  EM3D  dancing  arrows  and  providing  the  operator  with  a  range  of  test  items  to 
place  on  the  surface  to  practice  observing  the  dancing  arrow  responses  and  target  detection.  The 
approach  also  does  not  generate  a  data  map  or  record  of  the  dynamic  survey.  Even  in  spite  of 
these  limitations,  these  results  illustrate  that  classification  can  be  successful  in  areas  without 
accurate  positional  data. 
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